Skip to content

Latest commit

 

History

History
303 lines (233 loc) · 21.6 KB

FAQ.md

File metadata and controls

303 lines (233 loc) · 21.6 KB
Raw

Frequently Asked Questions

Content

General {.unnumbered}

Darwin Core {.unnumbered}

Formatting Data {.unnumbered}

Vocabulary {.unnumbered}

Taxonomy {.unnumbered}

  • How do I document occurrences from unknown species, those new to science, or those with temporary names? e.g. Eurythenes sp. DISCOLL.PAP.JC165.674

    Occurrences unknown or new to science should be documented according to recommendations by Horton et al. 2021. You should populate the scientificName field with the genus, and in identificationQualifer provide the ON sign 'sp.'. However you must also indicate the reason why species-level identification is unavailable. To do this, supplement 'sp.' with either stet. (stetit) or indet. (indeterminabilis). If neither of these are applicable, (e.g. for undescribed new species), add a unique taxon identifier code after 'sp.' to identificationQualifer. For example Eurythenes sp. DISCOLL.PAP.JC165.674.

    Please avoid simple alphanumeric codes (i.e. Eurythenes sp. 1, Eurythenes sp. A). Similar to creating eventIDs or occurrenceIDs, you should strive to provide more complex and globally unique identifier. Identifiers could be constructed by combining higher taxonomic information with information related to a collection, institution, museum or collection code, sample number or museum accession number, expedition, dive number, or timestamp. This ensures namestrings will remain unique within a larger repositories like OBIS. It is also recommended to include these temporary names on specimen labels for physical specimens.

  • Can scientificNameID be populated with an identifier (e.g. WoRMS LSID) representing an unaccepted taxon name?

    Yes. The identifier in scientificNameID should always correspond with the name that is in the scientificName field, even if the name is an unaccepted name in WoRMS. For example, the species name "Holothuria mammiculata" was provided, but this name is unaccepted in WoRMS. The accepted name is "Holothuria (Stauropora) pervicax Selenka, 1867". In this case scientificNameID should correspond to the original name with LSID urn:lsid:marinespecies.org:taxname:529968 because the ID must correlate with the name as recorded in scientificName.

  • What happens when a WoRMS taxonomy is changed? e.g. a species is reclassified

    When species are reclassied in WoRMS, the original scientificName and scientificNameID provided in a dataset remains unchanged. However WoRMS will list the old ID as "Unaccepted", and link to the accepted taxon entry, and this will be reflected in the taxonomic information attached to a dataset download.

    For example, if we search for Manta birostris in OBIS (https://obis.org/taxon/105857), we see that the taxon's status in WoRMS is unaccepted. At the bottom of the page it links to the currently accepted name: https://obis.org/taxon/1026118. We can find an occurrence which shows the source scientificName as "Manta" while the interpreted scientificName is "Mobula": https://obis.org/occurrence/00032775-252c-44b3-9c95-be1c9adb2fe4. We can also see that originalScientificName is populated with the source name in the intepreted output.

    As a user, you don't have to trace species names. However if the datasets's DwC-A is downloaded from the dataset page instead of obtained through R or the Mapper, all fields will contain the original value. It remains good practice to also check identifiers against WoRMS to see if any have been updated when you download data.

Tools {.unnumbered}

Quality Control {.unnumbered}

Publishing {.unnumbered}

Accessing data in OBIS {.unnumbered}

  • Can I get (or publish) other types of information from OBIS, besides occurences (e.g. bio-eco variables, DNA data, abiotic measurements, etc.)?

    Yes! OBIS allows access to not only taxa occurrences, but also all kinds of measurement data including DNA data (see Contribute for a list of data accepted by OBIS). This data is recorded in the (Extended) Measurement Or Fact and DNA-Derived Data extension tables, respectively. To access this data when downloading, be sure to select MoF or DNA from the dropdown Dataset Type menu on the advanced dataset serach page. If you are obtaining data from the OBIS Mapper, make sure to check the box for MeasurementOrFact and/or DNADerivedData from the Extensions toggle section. When using the R package robis, use the mof=TRUE and/or dna=TRUE arguments. More data filter options will be added in the future.

  • How do I download data from OBIS?
  • How do I load the full (.csv) export of OBIS data?

    Loading the entire OBIS dataset uses a lot of memory and is probably not feasible on most desktop computers. You have a few potential options depending on the use case: i) process the data in smaller batches, or ii) load the dataset into a local database such as SQLite and use SQL queries to analyze the data

    Otherwise, we recommend you use the parquet download which is available here, instead of the CSV. Then in R, you can use the arrow package to work with parquet files. We also have a short tutorial on working with parquet files in R here, with an example application of this approach here (see first code block).

  • How can I use R to access OBIS data?
  • How do I use the OBIS API to fetch and filter data?
  • How do I contact the data provider?
  • How can I cite OBIS datasets and downloads?
  • What are the definitions of the field names in the downloads generated by OBIS?
  • How do I obtain a taxon checklist for an area?

    There are a few possible ways to obtain a taxon checklist for a given area. We will obtain a checklist of species in the Albain EEZ as an example. To do this we will create a bounding box around our area of interest, and then apply filters to simplify the geometry.

    library(mregions)
library(dplyr)
library(robis)
library(sf)
#obtain Albanian EEZ as sf
geom <- mr_shp(key = "MarineRegions:eez", filter = "Albanian Exclusive Economic Zone", maxFeatures = NULL)
#get WKT for the bounding box
wkt <- st_as_text(st_as_sfc(st_bbox(geom)), digits = 6)
#fetch occurrences for bounding box
occ <- occurrence(geometry = wkt) %>%
  st_as_sf(coords = c("decimalLongitude", "decimalLatitude"), crs = 4326)
#filter using geometry
occ_filtered <- occ %>%
  filter(st_intersects(geometry, geom, sparse = FALSE)) %>%
  as_tibble() %>%
  select(-geometry)
#get taxa
alb_taxa <- occ_filtered %>%
  group_by(phylum, class, order, family, genus, species, scientificName) %>%
  summarize(records = n())
  • The dates look unusual in the download file. What are these, how do I convert them, and/or how do I obtain separate elements from them (e.g. month)?

    The values in date_start, date_mid, and date_end are unix timestamps which have been calculated from the ISO date in the eventDate column. We can convert these numerical values to dates using the formula below.

    =(E2/86400000)+DATE(1970,1,1)

    If, when you apply this formula, you still see numbers, you will need to set the cell formatting to Date. Once you have dates, you can obtain, e.g. months for seasonal analyses using:

    =MONTH(H2)

    You can also use this tool to convert timestamps.

  • How do I filter by or obtain trait information for OBIS data (e.g. all benthic organisms)?

    Currently, it is not possible to filter OBIS data by trait. To do this, we recommend using the traits database of the World Register of Marine Species. For example, searching by “functional group”, you can specify benthos, plankton, nekton, etc.

  • How do I get data from multiple regions from OBIS?

    If the areas OBIS currently uses does not work for your use case, then it is best to first define all the boundaries for the desired regions. OBIS can be queried using WKT polygons by providing a WKT string to the geometry parameter in the robis::occurrence function. HOWEVER there are some limitations with respect to polygon complexity, and if it is too complex you will likely receive the error "The OBIS API was not able to process your request".

    For more complex spatial queries we recommend indexing OBIS and GBIF data against polygons and using (finely) gridded versions of these datasets to make the process faster. We note we have not yet properly documented this process, but see the example script produced by Pieter Provoost below. The script first indexes a polygon to the H3 spatial index, then queries a gridded version of OBIS+GBIF data on AWS to get the species list, and finally fetches taxonomy from WoRMS for every species, which may take some time.

    library(readr)
library(h3jsr)
library(sf)
library(duckdb)
library(DBI)
library(dplyr)

sf_use_s2(FALSE)

# Read WKT from text file, convert to sf, and index to H3 resolution 7
# https://wktmap.com/?e6b28728

wkt <- read_file("wkt_21773.txt")
geom <- st_as_sfc(wkt, crs = 4326)
cells <- data.frame(cell = polygon_to_cells(geom, 7)[[1]])

# Set up duckdb connection and register cells table

con <- dbConnect(duckdb())
dbSendQuery(con, "install httpfs; load httpfs;")
duckdb_register(con, "cells", cells)

# Join cells list and gridded species dataset

species <- dbGetQuery(con, "
select species, AphiaID
from cells
inner join read_parquet('s3://obis-products/speciesgrids/h3_7/*') h3 on cells.cell = h3.h3_07
group by species, AphiaID
")

# Add WoRMS taxonomy

id_batches <- split(species$AphiaID, ceiling(seq_along(species$AphiaID) / 50))
taxa_batches <- purrr::map(id_batches, worrms::wm_record)
taxa <- bind_rows(taxa_batches) %>% 
select(AphiaID, scientificname, phylum, class, order, family, genus, scientificName = scientificname)

# Get Mollusca species

mollusca <- taxa %>%
filter(phylum == "Mollusca")