Mapping of checks in obis‐qc to TDWG BDQ core tests and assertions

The OBIS Data Quality Control Project Team (QCPT) has mapped the checks performed in obis-qc to the Core Tests and Assertions developed by TDWG Biodiversity Data Quality (BDQ) Task Group 2.

This page serves to record all the decisions made during the course of this project team for the tests and parameters. All parameters and checks mentioned in this page is a snapshot on 2023-09-14. Please note that even though the test parameters have been adjusted during the course of this project team, they may not be implemented yet. The table below outlined what has been decided by the project team, but the implementation depends on the maintainer of obis-qc.

Definition of the fields and vocabulary used can be found in this page.

obis-qc_github_issue	obis-qc_information_element	obis-qc_description	obis-qc_flag	bdq_test_label	bdq_github_issue	bdq_param	obis-qc_param	bdq_source_authority	obis-qc_source_authority	remarks
https://github.com/iobis/obis-qc/issues/23	occurrenceStatus	occurrenceStatus should be present.	missing_occurrenceStatus	VALIDATION_OCCURRENCESTATUS_NOTEMPTY	https://github.com/tdwg/bdq/issues/117
https://github.com/iobis/obis-qc/issues/23	occurrenceStatus	occurrenceStatus should be absent or present.	invalid_occurrenceStatus	VALIDATION_OCCURRENCESTATUS_STANDARD	https://github.com/tdwg/bdq/issues/116	dwc:occurrenceStatus vocabulary	dwc:occurrenceStatus vocabulary	bdq:sourceAuthority = "Darwin Core occurrenceStatus" {https://dwc.tdwg.org/list/#dwc_occurrenceStatus} {dwc:occurrenceStatus vocabulary [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}	obis:sourceAuthority = "Darwin Core occurrenceStatus" {https://dwc.tdwg.org/list/#dwc_occurrenceStatus} {dwc:occurrenceStatus vocabulary [https://rs.gbif.org/vocabulary/gbif/occurrence_status_2020-07-15.xml]}	obis-qc does not amend the value of occurrenceStatus, rather it sets a custom field "absence" to true if occurrenceStatus == "absent".
https://github.com/iobis/obis-qc/issues/23	individualCount	If individualCount equals 0, record is absence.	invalid_occurrenceStatus							obis-qc does not amend the value of occurrenceStatus, rather it sets a custom field "absence" to true if individualCount == 0. Mismatch occurrenceStatus value with individualCount will lead to occurrenceStatus being flagged invalid.
	basisOfRecord	basisOfRecord should be present.	missing_basisOfRecord	VALIDATION_BASISOFRECORD_NOTEMPTY	https://github.com/tdwg/bdq/issues/58
https://github.com/iobis/obis-qc/issues/37	basisOfRecord	basisOfRecord should be one of the values in vocabulary specified in obis:sourceAuthority		VALIDATION_BASISOFRECORD_STANDARD	https://github.com/tdwg/bdq/issues/104	dwc:basisOfRecord vocabulary	dwc:basisOfRecord vocabulary	bdq:sourceAuthority default = "Darwin Core basisOfRecord" {[https://dwc.tdwg.org/terms/#dwc:basisOfRecord]}{dwc:basisOfRecord vocabulary [https://rs.gbif.org/vocabulary/dwc/basis_of_record.xml]}	obis:sourceAuthority default = "Darwin Core basisOfRecord" {[https://dwc.tdwg.org/terms/#dwc:basisOfRecord]}{dwc:basisOfRecord vocabulary [https://rs.gbif.org/vocabulary/dwc/basis_of_record.xml]}
https://github.com/iobis/obis-qc/issues/24	eventDate	eventDate should be present.	missing_eventDate	VALIDATION_EVENTDATE_NOTEMPTY	https://github.com/tdwg/bdq/issues/33
https://github.com/iobis/obis-qc/issues/24	eventDate	eventDate should conform to ISO 8601.	invalid_eventDate	VALIDATION_EVENTDATE_STANDARD	https://github.com/tdwg/bdq/issues/66
https://github.com/iobis/obis-qc/issues/24	eventDate	eventDate should not be in the future.	DATE_IN_FUTURE	VALIDATION_EVENTDATE_INRANGE	https://github.com/tdwg/bdq/issues/36	bdq:earliestValidDate | bdq:latestValidDate	obis:earliestValidDate	bdq:earliestValidDate default="1582-11-15" | bdq:latestValidDate default=current year	obis:earliestValidDate=1582
https://github.com/iobis/obis-qc/issues/24	eventDate	eventDate should be later than the obis:earliestValidDate	DATE_BEFORE_MIN	VALIDATION_EVENTDATE_INRANGE	https://github.com/tdwg/bdq/issues/36	bdq:earliestValidDate | bdq:latestValidDate	obis:latestValidDate	bdq:earliestValidDate default="1582-11-15" | bdq:latestValidDate default=current year	obis:latestValidDate=today
https://github.com/iobis/obis-qc/issues/24	year	year should be within range	YEAR_NOT_IN_RANGE	VALIDATION_YEAR_INRANGE	https://github.com/tdwg/bdq/issues/84	bdq:earliestValidDate | bdq:latestValidDate	obis:earliestValidDate | obis:latestValidDate	bdq:earliestValidDate="1582" | bdq:latestValidDate=current year	obis:earliestValidDate = 1582 | obis:latestValidDate = current year	prerequisite for AMENDMENT_EVENTDATE_FROM_YEARMONTHDAY
https://github.com/iobis/obis-qc/issues/24	month	month should be an integer between 1 and 12 inclusive	invalid_month	VALIDATION_MONTH_STANDARD	https://github.com/tdwg/bdq/issues/126
https://github.com/iobis/obis-qc/issues/24	eventDate, year, month, day	if eventDate is empty and a valid ISO eventDate can be constructed from the values of year, month, day, construct and populate eventDate.	EVENTDATE_FROM_YEARMONTHDAY	AMENDMENT_EVENTDATE_FROM_YEARMONTHDAY	https://github.com/tdwg/bdq/issues/93
https://github.com/iobis/obis-qc/issues/25	decimalLongitude	decimalLongitude should be present.	NO_COORD	VALIDATION_DECIMALLONGITUDE_NOTEMPTY	https://github.com/tdwg/bdq/issues/96
https://github.com/iobis/obis-qc/issues/25	decimalLatitude	decimalLatitude should be present.	NO_COORD	VALIDATION_DECIMALLATITUDE_NOTEMPTY	https://github.com/tdwg/bdq/issues/119
https://github.com/iobis/obis-qc/issues/25	decimalLongitude, decimalLatitude	decimalLatitude and decimalLongitude should not be zero.	ZERO_COORD	VALIDATION_COORDINATES_NOTZERO	https://github.com/tdwg/bdq/issues/87					obis-qc requires both decimalLongitude == 0 AND decimalLatitude == 0 to fail the test and be flagged as ZERO_COORD; bdq core test will return "not compliant" when decimalLongitude == 0 OR decimalLatitude == 0
https://github.com/iobis/obis-qc/issues/25	decimalLongitude	decimalLongitude should be within range. The range of decimalLongitude is between -180 to 180.	LON_OUT_OF_RANGE, NO_COORD	VALIDATION_DECIMALLONGITUDE_INRANGE	https://github.com/tdwg/bdq/issues/30
https://github.com/iobis/obis-qc/issues/25	decimalLatitude	decimalLatitude and decimalLongitude should be within range. The range of decimalLatitude is between -90 and 90.	LAT_OUT_OF_RANGE, NO_COORD	VALIDATION_DECIMALLATITUDE_INRANGE	https://github.com/tdwg/bdq/issues/79
	coordinateUncertaintyInMeters	coordinateUncertaintyInMeters should be present.	missing_coordinateUncertaintyInMeters
https://github.com/iobis/obis-qc/issues/36	coordinateUncertaintyInMeters	coordinateUncertaintyInMeters should be within the range of 1 and 20,037,509.	invalid_coordinateUncertaintyInMeters	VALIDATION_COORDINATEUNCERTAINTY_INRANGE	https://github.com/tdwg/bdq/issues/109
https://github.com/iobis/obis-qc/issues/22	minimumDepthInMeters, maximumDepthInMeters	minimumDepthInMeters and maximumDepthInMeters should be present.	NO_DEPTH
https://github.com/iobis/obis-qc/issues/22	minimumDepthInMeters	minimumDepthInMeters should be within obis:minimumValidDepthInMeters and obis:maximumValidDepthInMeters.	DEPTH_OUT_OF_RANGE	VALIDATION_MINDEPTH_INRANGE	https://github.com/tdwg/bdq/issues/107	bdq:minimumValidDepthInMeters | bdq:maximumValidDepthInMeters	obis:minimumValidDepthInMeters | obis:maximumValidDepthInMeters	bdq:minimumValidDepthInMeters default="0" | bdq:maximumValidDepthInMeters default="11000"	obis:minimumValidDepthInMeters default="0" | obis:maximumValidDepthInMeters default="11000"
https://github.com/iobis/obis-qc/issues/22	maximumDepthInMeters	maximumDepthInMeters should be within obis:minimumValidDepthInMeters and obis:maximumValidDepthInMeters.	DEPTH_OUT_OF_RANGE	VALIDATION_MAXDEPTH_INRANGE	https://github.com/tdwg/bdq/issues/187	bdq:minimumValidDepthInMeters | bdq:maximumValidDepthInMeters	obis:minimumValidDepthInMeters | obis:maximumValidDepthInMeters	bdq:minimumValidDepthInMeters default="0" | bdq:maximumValidDepthInMeters default="11000"	obis:minimumValidDepthInMeters default="0" | obis:maximumValidDepthInMeters default="11000"
https://github.com/iobis/obis-qc/issues/22	minimumDepthInMeters, maximumDepthInMeters	minimumDepthInMeters should not exceeds maximumDepthInMeters.	MIN_DEPTH_EXCEEDS_MAX	VALIDATION_MINDEPTH_LESSTHAN_MAXDEPTH	https://github.com/tdwg/bdq/issues/24
https://github.com/iobis/obis-qc/issues/22	minimumDepthInMeters	minimumDepthInMeters should be less than or equal to the obis:bathymetricMaxDepth for European water body and obis:bufferedBathymetricMeanDepth for other water body.	DEPTH_EXCEEDS_BATH	VALIDATION_MAXDEPTH_INRANGE	https://github.com/tdwg/bdq/issues/107	bdq:minimumValidDepthInMeters | bdq:maximumValidDepthInMeters	obis:bufferedBathymetricMeanDepth | obis:bathymetricMaxDepth		global water body = GEBCO bathymetry layer, European water body = EMODnet bathymetry layer | global water bufferedBathymetricMeanDepth = slope*mean bathymetric depth of grid + intercept where slope = 1.1, intercept = 50 see: https://github.com/iobis/obis-qc/issues/14; European water bathymetricMaxDepth = maximum depth of bathymetry grid
https://github.com/iobis/obis-qc/issues/22	maximumDepthInMeters	minimumDepthInMeters should be less than or equal to the obis:bathymetricMaxDepth for European water body and obis:bufferedBathymetricMeanDepth for other water body.	DEPTH_EXCEEDS_BATH	VALIDATION_MINDEPTH_INRANGE	https://github.com/tdwg/bdq/issues/187	bdq:minimumValidDepthInMeters | bdq:maximumValidDepthInMeters	obis:bufferedBathymetricMeanDepth | obis:bathymetricMaxDepth		global water body = GEBCO bathymetry layer, European water body = EMODnet bathymetry layer | global water bufferedBathymetricMeanDepth = slope*mean bathymetric depth of grid + intercept where slope = 1.1, intercept = 50 see: https://github.com/iobis/obis-qc/issues/14; European water bathymetricMaxDepth = maximum depth of bathymetry grid
https://github.com/iobis/obis-qc/issues/26	decimalLongitude, decimalLatitude	Is the occurrence located on land? Based on OpenStreetMap land polygons.	ON_LAND	VALIDATION_COORDINATES_TERRESTRIALMARINE	https://github.com/tdwg/bdq/issues/51	bdq:taxonIsMarine | bdq:geospatialLand | bdq:spatialBufferInMeters	obis:taxonNotExclusivelyFreshwaterOrTerrestrial | obis:geospatialLand | obis:spatialBufferInMeters	bdq:taxonIsMarine default = "World Register of Marine Organisms (WoRMS") {[https://www.marinespecies.org/]} {Web service [https://www.marinespecies.org/aphia.php?p=webservice]} | {bdq:geospatialLand default = The spatial union of "NaturalEarth 10m-physical-vectors for Land" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip] and "NaturalEarth Minor Islands" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_minor_islands.zip]} | bdq:spatialBufferInMeters default = "3000"	obis:taxonNotExclusivelyFreshwaterOrTerrestrial = "World Register of Marine Organisms (WoRMS") {[https://www.marinespecies.org/]} {Web service [https://www.marinespecies.org/aphia.php?p=webservice]} | obis:geospatialLand = OpenStreetMap land polygon [https://osmdata.openstreetmap.de/data/land-polygons.html]} | obis:spatialBufferInMeters default = "3000"	meeting decided to expand OBIS scope. Records with taxa which are exclusively freshwater or exclusively terrestrial that are located on land will not be ingested.
https://github.com/iobis/obis-qc/issues/34	scientificName, scientificNameID	Taxon should not be exclusively freshwater or terrestrial according to WoRMS.	NOT_MARINE, MARINE_UNSURE	VALIDATION_COORDINATES_TERRESTRIALMARINE	https://github.com/tdwg/bdq/issues/51	bdq:taxonIsMarine | bdq:geospatialLand | bdq:spatialBufferInMeters	obis:taxonNotExclusivelyFreshwaterOrTerrestrial | obis:geospatialLand | obis:spatialBufferInMeters	bdq:taxonIsMarine default = "World Register of Marine Organisms (WoRMS") {[https://www.marinespecies.org/]} {Web service [https://www.marinespecies.org/aphia.php?p=webservice]} | {bdq:geospatialLand default = The spatial union of "NaturalEarth 10m-physical-vectors for Land" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_land.zip] and "NaturalEarth Minor Islands" [https://www.naturalearthdata.com/http//www.naturalearthdata.com/download/10m/physical/ne_10m_minor_islands.zip]} | bdq:spatialBufferInMeters default = "3000"	obis:taxonNotExclusivelyFreshwaterOrTerrestrial = "World Register of Marine Organisms (WoRMS") {[https://www.marinespecies.org/]} {Web service [https://www.marinespecies.org/aphia.php?p=webservice]} | obis:geospatialLand = OpenStreetMap land polygon [https://osmdata.openstreetmap.de/data/land-polygons.html]} | obis:spatialBufferInMeters default = "3000"	meeting decided to expand OBIS scope. Records with taxa which are exclusively freshwater or exclusively terrestrial that are located on land will not be ingested.
https://github.com/iobis/obis-qc/issues/34	scientificName	scientificName should be present.	missing_scientificName	VALIDATION_SCIENTIFICNAME_NOTEMPTY	https://github.com/tdwg/bdq/issues/82
https://github.com/iobis/obis-qc/issues/34	scientificNameID	scientificNameID should be present.	missing_scientificNameID
https://github.com/iobis/obis-qc/issues/34	scientificNameID	scientificNameID should be valid WoRMS LSID.	invalid_scientificNameID						obis:sourceAuthority default = "World Register of Marine Species" {[https://www.marinespecies.org/]} {API endpoint [https://www.marinespecies.org/rest/]}
https://github.com/iobis/obis-qc/issues/34	scientificName, scientificNameID	Taxon should unambiguously match with WoRMS.	NO_MATCH	VALIDATION_SCIENTIFICNAME_FOUND	https://github.com/tdwg/bdq/issues/46	bdq:sourceAuthority	obis:sourceAuthority	bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]}	obis:sourceAuthority default = "World Register of Marine Species" {[https://www.marinespecies.org/]} {API endpoint [https://www.marinespecies.org/rest/]}
https://github.com/iobis/obis-qc/issues/34	scientificName, scientificNameID	Taxon should unambiguously match with WoRMS.	NO_MATCH	VALIDATION_TAXON_UNAMBIGUOUS	https://github.com/tdwg/bdq/issues/70	bdq:sourceAuthority	obis:sourceAuthority	bdq:sourceAuthority default = "GBIF Backbone Taxonomy" {[https://doi.org/10.15468/39omei]} {API endpoint [https://api.gbif.org/v1/species?datasetKey=d7dddbf4-2cf0-4f39-9b2a-bb099caae36c&name=]}	obis:sourceAuthority default = "World Register of Marine Species" {[https://www.marinespecies.org/]} {API endpoint [https://www.marinespecies.org/rest/]}
https://github.com/iobis/obis-qc/issues/34	scientificName, scientificNameID	An accepted name should exist in WoRMS.	NO_ACCEPTED_NAME
	measurementType, measurementTypeID	measurementType and measurementTypeID should be present if extended Measurement or Fact extension is being used.