Skip to content

Commit

Permalink
minor formatting fixes
Browse files Browse the repository at this point in the history
* changed image sizes to  make them larger
* added some individual page Table of Contents
* fixed some image captions
  • Loading branch information
EliLawrence committed May 11, 2023
1 parent 8a5b622 commit 9b90f70
Show file tree
Hide file tree
Showing 10 changed files with 72 additions and 57 deletions.
8 changes: 4 additions & 4 deletions access.md
Original file line number Diff line number Diff line change
Expand Up @@ -83,15 +83,15 @@ From the OBIS homepage, you can search for data in the search bar in the middle

When you search by dataset you will notice an additional option appears for [advanced search options](https://obis.org/datasets). This will allow you to identify specific datasets, and apply filters for OBIS nodes and whether datasets include extensions.

![*OBIS homepage search, showing where to find the advanced search link*](images/obis-homepagesearch.png){width=70%}
![*OBIS homepage search, showing where to find the advanced search link*](images/obis-homepagesearch.png){width=90%}

Regardless if you found a dataset through the homepage or the advanced Dataset search, you will be able to navigate to individual dataset pages. For individual dataset pages (instead of aggregate pages for e.g., a Family) there are three buttons available:

* Report issue - allows you to report any issues with the dataset in question
* Source DwC-A - download the dataset as a Darwin Core-Archive file. This will provide all data tables as separate files within a zipped folder
* To mapper - this will open another browser with the data shown in the Mapper

![*Dataset download*](images/dataset-DL.png){width=70%}
![*Dataset download*](images/dataset-DL.png){width=90%}

If you searched for aggregate datasets (e.g., all Crustacea records, all records from OBIS-Canada, etc.), the `source DwC-A` button will not be available to you. To download these data subsets, you must click `to mapper` and then [download the data from the Mapper as a CSV](#mapper).

Expand All @@ -101,11 +101,11 @@ If you searched for aggregate datasets (e.g., all Crustacea records, all records

To obtain a full export of OBIS data, navigate to the OBIS homepage, click on Data from the top navigation bar, then select [Data Access](https://obis.org/data/access/) from the dropdown menu.

![*OBIS homepage showing where to navigate to access full database exports*](images/full-export1.png){width=70%}
![*OBIS homepage showing where to navigate to access full database exports*](images/full-export1.png)

Here you will be able to download all occurrence records as a CSV or Parquet file. Note the disclaimer that such exports will not include measurement data, dropped records, or absence records. As with downloads from the Mapper, the exported file is a single Occurrence table. This table includes all provided Event and Occurrence data, as well as 68 fields added by the OBIS Quality Control Pipeline, including taxonomic information obtained from WoRMS.

![*OBIS Data Access page*](images/full-export2.png){width=70%}
![*OBIS Data Access page*](images/full-export2.png)

## Finding your own data in OBIS

Expand Down
22 changes: 12 additions & 10 deletions common_formatissues.md
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,9 @@

- [Missing required fields](#missing-required-fields)
- [Temporal issues: dates/times](#temporal-dates-and-times)
- [Spatial issues: coordinates, geographical formats](#spatial)
- [Historical data](#historical-data)
- [Spatial issues: coordinate conversion](#spatial)
- [Convert geographical formats](#geographical-format-conversion)

### Missing required fields

Expand Down Expand Up @@ -105,15 +107,15 @@ For uncertainty regarding the date of the event, see [guidelines](common_qc.html

To ensure your date is formatted correctly, it may be easiest to begin by populating the `year`, `month`, and `day` fields first. If the specific time of sampling is known, populate that into `eventTime` as well. When you fill these fields, we recommend ensuring the numbers are encoded as Text, not as General or numeric as Excel often tries to interpret what it thinks the content “should” be. Otherwise you may run into problems with Excel auto formatting your numbers in ways you don’t want. You can do this by highlighting the cells of interest, navigating to the Number Format on the Home ribbon and selecting “Text”. Be careful when you do this change of format, as some columns (e.g. time) may become formatted into a decimal or other unexpected format.

![Screenshot of how to change data type in Excel](images/excel-text-format.png){width=50%}
![*Screenshot of how to change data type in Excel*](images/excel-text-format.png){width=70%}

Then you can use Excel to concatenate each field together, adding the time zone at the end, using the general format:

```Excel
=CONCAT(YEAR, "-", MONTH, "-", DAY, "T", EVENTTIME, TIMEZONE)
```

![Example of how to concatenate dates in Excel](images/excel-concatexample.png){width=40%}
![*Example of how to concatenate dates in Excel*](images/excel-concatexample.png){width=70%}

> Note
> You can also use the Canadensys [date parsing](https://data.canadensys.net/tools/dates) tool to help you convert dates or parse them into component parts.
Expand Down Expand Up @@ -153,7 +155,7 @@ Our example dataset would then look like the following:

You can see that the eventDate for the parent events does not need to be provided - only the dates for the actual samples are required.

#### Historical data
### Historical data

Formatting historical data (data published before 1583 CE) can pose additional challenges due to restraints with the ISO 8601 standards. Namely, the shift from the Julian calendar to the currently used Gregorian calendar, as well as issues arising from the definition of [Year Zero](https://en.wikipedia.org/wiki/Year_zero).

Expand All @@ -169,7 +171,7 @@ More specific guidelines to address historical data complications are under deve

All coordinates provided in the `decimalLatitude` or `decimalLongitude` fields in OBIS must be in decimal degrees. To convert coordinates from degrees-minutes-seconds into decimal degrees, you can use [this Coordinate Conversion tool](https://obis.shinyapps.io/coordinates/) that OBIS has developed. This tool will convert any coordinate (or list of coordinates on a separate line) in a degrees-minutes-seconds format into decimal degrees, even partial coordinates. To use it, simply copy and paste your coordinates into the box provided and click Convert. For example:

![Screenshot of how to use the OBIS coordinate converter](images/coordinate_conversion.png){width=40%}
![*Screenshot of how to use the OBIS coordinate converter*](images/coordinate_conversion.png){width=70%}

The [Map Tool tutorial](access.html#mapper) also reviews use of the coordinate conversion tool.

Expand All @@ -183,23 +185,23 @@ In OBIS, the spatial reference system to be documented in `geodeticDatum` is [EP

You can load a .csv file containing your coordinates to be reprojected into [QGIS](https://qgis.org/en/site/forusers/download.html). Opening a new project, first set the global projection to WGS84 EPSG:4326. In the bottom right corner, click the Project Properties to change the Project Coordinate Reference System (CRS). A pop up window will allow you to search for and select WGS84 EPSG:4326. Click OK.

![Screenshot of QGIS interface](images/qgis_screenshot1.png){width=60%}
![*Screenshot of QGIS interface*](images/qgis_screenshot1.png){width=70%}

To load your .csv file containing the longitude and latitude coordinates, go to Layer < Add Layer < Add Delimited Text layer...

![How to add a .csv with coordinate data in QGIS](images/qgis_screenshot2.png){width=50%}
![*How to add a .csv with coordinate data in QGIS*](images/qgis_screenshot2.png){width=70%}

A popup window will allow you to browse and select your .csv file. Open the `Geometry Definition` portion of the window and map the field containing longitude values to the `X field` and latitude to the `Y field`. Select the CRS that these coordinates were recorded as from the drop down menu. Then click `Add` and close the window.

![Screenshot showing how to specify CRS of a .csv file when importing into QGIS](images/qgis_screenshot3.png){width=50%}
![*Screenshot showing how to specify CRS of a .csv file when importing into QGIS*](images/qgis_screenshot3.png){width=70%}

Go to Vector < Geometry Tools < Add Geometry Attributes

![Screenshot showing where to find the Geometry Attributes in QGIS menu](images/qgis_screenshot4.png){width=30%}
![*Screenshot showing where to find the Geometry Attributes in QGIS menu*](images/qgis_screenshot4.png){width=70%}

Make sure the input layer is your coordinate file. Under the `Calculate using`, select Project CRS (because we set the Project CRS to the desired projection). Click `Run`. This will create a new layer with an additional two columns called Xcoord (longitude) and Ycoord (latitude). These fields contain the coordinates in the desired projection (i.e., WGS84). You can view these columns by right clicking and opening the layer’s attribute table. To export the file, right click the layer and click Make Permanent. Then save the .csv.

![Screenshot showing how to save a temporary layer in QGIS for export](images/qgis_screenshot5.png){width=40%}
![*Screenshot showing how to save a temporary layer in QGIS for export*](images/qgis_screenshot5.png){width=40%}

For more details see this [QGIS guide on reprojection](https://docs.qgis.org/3.22/en/docs/training_manual/processing/crs.html?highlight=reproject).

Expand Down
32 changes: 16 additions & 16 deletions common_qc.md
Original file line number Diff line number Diff line change
Expand Up @@ -57,36 +57,36 @@ Please see the [video tutorial](access.html#mapper) on how to use our Map tool.

For both the [Getty thesaurus](https://www.getty.edu/research/tools/vocabularies/tgn/) and [Google Maps](https://www.google.com/maps/) you can simply search the name of a locality, for example the Cook Strait in New Zealand. The search result on the Getty thesaurus will bring you to a page where you can obtain `decimalLatitude` and `decimalLongitdue`.

![Screenshot of Cook Strait page on the Getty Thersaurus](images/getty-thesaurus-NZexample.png){width=60%}
![*Screenshot of Cook Strait page on the Getty Thersaurus*](images/getty-thesaurus-NZexample.png){width=80%}

For Google Maps, the coordinates can be found in the url after searching.

![Screenshot of Google Maps showing where coordinates can be foun din the URL](images/google-maps-coordinates.png){width=60%}
![*Screenshot of Google Maps showing where coordinates can be found in the URL*](images/google-maps-coordinates.png){width=80%}

### How to use Marine Regions Gazetteer tool

Marine Regions offers a marine gazetteer search engine to obtain geographic information and unique identifiers for marine regions. Once you have navigated to the [gazetteer search engine](https://www.marineregions.org/gazetteer.php?p=search), you have two options to search by: enter the name of the desired locality, or enter an MRGID code. Most likely you will have a locality name but not an MRGID. You may also select a [placetype](https://www.marineregions.org/gazetteer.php?p=popup&name=placetype) to search instead for types of regions that may be physical (e.g., seamount, bay, fjord, etc.) or administrative (e.g., exclusive economic zones, countries, etc). You can specify specific sources if known (e.g., published paper, organization, etc.). Finally, you can give a latitude/longitude coordinate with a radius around it to obtain a list of regions near that point.

For this example we will search by geographic name for the Bay of Fundy.

![Marine regions gazetteer search](images/marinegazeeteer-search.png){width=50%}
![Marine regions gazetteer search](images/marinegazeeteer-bayfundy.png){width=60%}
![ ](images/marinegazeeteer-search.png){width=80%}
![*Marine regions gazetteer search*](images/marinegazeeteer-bayfundy.png){width=80%}

Our search returned 5 results from different sources (indicated in brackets). So how do we select the correct one? We can notice right away that the second result, from SeaVox SeaArea, has a preferred alternative, which when you click on the link brings you to the IHO Sea Area description for Bay of Fundy. So already we can likely drop SeaVox as a potential candidate. A good next step may be to compare the geographical extent for each to ensure it covers the desired area. If you are uncertain about exactly where your locality is, it may be better to be safe and choose a wider geographic region. Let’s compare the maps for all 5 results:

![Bay of Fundy source IHO](images/marinegazeeteer-bayfundy-1-IHO.png){width=450%}
![Bay of Fundy source SeaVoX](images/marinegazeeteer-bayfundy-2-SeaVoX.png){width=50%}
![Canadian part of Bay of Fundy source Marine Region](images/marinegazeeteer-bayfundy-3-MarRegion.png){width=50%}
![Bay of Fundy source MEOW](images/marinegazeeteer-bayfundy-4-MEOW.png){width=50%}
![United States part of Bay of Fundy source Marine Region](images/marinegazeeteer-bayfundy-5-MarRegion.png){width=50%}
![*Bay of Fundy source IHO*](images/marinegazeeteer-bayfundy-1-IHO.png){width=50%}
![*Bay of Fundy source SeaVoX*](images/marinegazeeteer-bayfundy-2-SeaVoX.png){width=50%}
![*Canadian part of Bay of Fundy source Marine Region*](images/marinegazeeteer-bayfundy-3-MarRegion.png){width=50%}
![*Bay of Fundy source MEOW*](images/marinegazeeteer-bayfundy-4-MEOW.png){width=50%}
![*United States part of Bay of Fundy source Marine Region*](images/marinegazeeteer-bayfundy-5-MarRegion.png){width=50%}

Notice that no region has the exact same geographic extent. Let’s select the IHO Bay of Fundy locality (the first search result) to ensure we are covering the entire area of the Bay of Fundy, but not the Gulf. Inspecting the rest of the page, there is a lot of other useful information we can use. We can populate the following OBIS fields for our dataset, copying the information outlined in the red boxes:

1. `locationID` from MRGID: <http://marineregions.org/mrgid/4289>
2. `decimalLatitude` and `decimalLongitude` latitude and longitude coordinates of the location’s midpoint in decimal degrees: 44.97985204, -65.80601556
3. `coordinateUncertaintyInMeters` precision: 196726 meters

![Screenshot of Marine Region placedetails, highlighting important information for OBIS](images/MarineRegions-Gazetteer_placedetails.png){width=40%}
![*Screenshot of Marine Region placedetails, highlighting important information for OBIS*](images/MarineRegions-Gazetteer_placedetails.png){width=80%}

Since we are obtaining all this locality data from Marine Regions, we must also populate the `locationAccordingTo` field. Here, we will provide the name of the gazetteer we used to obtain the coordinates for the locality - in this case you would write “Marine Regions”. In `georeferenceRemarks` we must document that the coordinates are the region’s midpoint, that locality information was inferred by geographic name, and, where applicable, place the original locality name in the field `verbatimLocality`. Finally, the location portion of our dataset would look something like this:

Expand Down Expand Up @@ -130,9 +130,9 @@ There is a new Darwin Core term [`verbatimIdentification`](https://dwc.tdwg.org/

The use and definitions for additional Open Nomenclature (ON) signs (`identificationQualifier`) can be found in [Open Nomenclature in the biodiversity era](https://doi.org/10.1111/2041-210X.12594), which provides examples for using the main Open Nomenclature qualifiers associated with physical specimens (Figure 1). Whereas the publication [Recommendations for the Standardisation of Open Taxonomic Nomenclature for Image-Based Identifications](https://www.frontiersin.org/articles/10.3389/fmars.2021.620702/full) provides examples and definitions for identificationQualifiers for non-physical specimens (image-based) (Figure 2).

![Figure 1. Flow diagram with the main Open Nomenclature qualifiers associated with physical specimens. The degree of confidence in the correct identifier increases from the top down. More info and figure copied from [Open Nomenclature in the biodiversity era](https://doi.org/10.1111/2041-210X.12594).](images/fig1-openNomenclature.png){width=60%}
![*Figure 1. Flow diagram with the main Open Nomenclature qualifiers associated with physical specimens. The degree of confidence in the correct identifier increases from the top down. More info and figure copied from [Open Nomenclature in the biodiversity era](https://doi.org/10.1111/2041-210X.12594).*](images/fig1-openNomenclature.png){width=80%}

![Figure 2: Flow diagram with the main Open Nomenclature qualifiers for the identification of specimens from images (non-physical, image-based) . More information and figure copied from [Recommendations for the Standardisation of Open Taxonomic Nomenclature for Image-Based Identifications](https://www.frontiersin.org/articles/10.3389/fmars.2021.620702/full)](images/fig2-flowDiagram.jpg){width=60%}
![*Figure 2: Flow diagram with the main Open Nomenclature qualifiers for the identification of specimens from images (non-physical, image-based) . More information and figure copied from [Recommendations for the Standardisation of Open Taxonomic Nomenclature for Image-Based Identifications](https://www.frontiersin.org/articles/10.3389/fmars.2021.620702/full)*](images/fig2-flowDiagram.jpg){width=80%}

#### Changes in taxonomic classification

Expand Down Expand Up @@ -171,18 +171,18 @@ Otherwise, records marked as non-marine will be dropped from the published datas

Let’s consider an example within [this dataset](https://obis.org/dataset/9fbaeb21-a0dc-4a29-8237-1cd7ada266e0) on benthic macroalgae. Inspecting the data quality report we can see there are three dropped records due to species not being marine.

![Dropped records from a benthic macroalgae dataset](images/dropped-records1.png){width=40%}
![*Dropped records from a benthic macroalgae dataset*](images/dropped-records1.png){width=80%}

Clicking on the dropped records we can see which three species were dropped. By scrolling to the right of the table, we can see these records have two quality flags: NO_DEPTH and NOT_MARINE.

![Flags specifying why certain records were dropped](images/dropped-records2.png){width=40%}
![*Flags specifying why certain records were dropped*](images/dropped-records2.png){width=80%}

Let’s take a look at the first species, Pseudochantransia venezuelensis. When we search for this species on [WoRMS](https://www.marinespecies.org/aphia.php?p=taxdetails&id=836900) we can see that the species is marked as freshwater.

![ ](images/nonmarine-spp-worms.png){width=60%}
![ ](images/nonmarine-spp-worms.png){width=80%}

Cross-referencing with IRMNG, if we search for the genus-species, the species is not even found, an indication that it is not in the database (and also why it can be good to check multiple sources). Searching for just the genus, we can see that marine and brackish are stricken out, indicating the species is not marine.

![ ](images/nonmarine-spp-irmng.png){width=60%}
![ ](images/nonmarine-spp-irmng.png){width=80%}

If you have species that are marked as non-marine in these registers but are either supposed to be marine, or were found in a marine environment, then you should contact WoRMS to discuss adding it to the register. For additions and/or edits to environmental or distribution records of a species, contact the WoRMS Data Management Team at [email protected] with your request along with your record or publication substantiating the addition/change.
Loading

0 comments on commit 9b90f70

Please sign in to comment.