Merge pull request #440 from monarch-initiative/add-externally-manage…

…d-content

Add pipeline for externally managed content

docs/external/nord.md

@@ -0,0 +1,33 @@
+# NORD - National Organization for Rare Disorders
+
+**Source name:** NORD - Externally managed content
+
+**Source description:** 
+
+> NORD advances practical, meaningful, and enduring change so people with rare diseases can live their fullest and best lives. Every day, we elevate care, advance research, and drive policy in a purposeful and holistic manner to lift up the rare disease community. (https://rarediseases.org/, 22.02.2024)
+
+NORD provides three datatypes to us:
+
+* Cross-references to  NORD content
+* Subset-declarations, which basically correspond to "what NORD considers a Rare Disease"
+* Preferred Names for certain diseases
+
+The content is provided by an API endpoint at NORD:
+
+```
+https://rdbdev.wpengine.com/wp-content/uploads/mondo-export/rare_export.tsv
+```
+
+For additional information, see [mondo-ingest.Makefile](https://github.com/monarch-initiative/mondo-ingest/blob/main/src/ontology/mondo-ingest.Makefile), in particular the goal called `$(TMPDIR)/nord.tsv:`.
+
+
+**Homepage:** https://rarediseases.org/
+
+**Comments about this source:** 
+
+The pipeline works like this:
+
+1. The TSV is downloaded from the NORD endpoint
+2. A script injects a ROBOT template header into  the TSV and then compiles it into OWL.
+3. On the Mondo repo side, we have a script that deletes old NORD content, and merges in the updated version.
+4. As of 22.02.2024, the NORD TSV file needs to be manually updated on the NORD side. This does not affect us, but could explain why certain things are not updated in sync with their website. NORD knows about this and tries to find a solution.

docs/externally-managed-content.md

@@ -0,0 +1,19 @@
+## Externally managed content
+
+Externally managed content is content that is provided by trusted providers and is merged in _unreviewed_. Currently, we support three types of externally managed content:
+
+1. Preferred names / labels. If a partner organisation of Monarch has a certain preference for a name this can be recorded as part of the metadata.
+2. Cross-references and linkouts. Partner organisations can provide cross-references and linkout to important resources related to a disease.
+3. Subsets. Partner organisations can provide subset information to Mondo. This is used in a variety of ways, such as:
+   - NORD declares which diseases it consideres "rare"
+   - Open Targets declares which diseases are used for their drug-prediction framework
+
+### Typical workflows
+
+1. External provider provides a TSV. (Ideally they use the same template that NORD uses - see `src/ontology/external/nord.robot.tsv`).
+2. We pull it in and turn it into a ROBOT template and transform it to owl.
+
+### Related issues and PRs:
+
+- [Issue: Represent Externally Managed Content in the Mondo Ingest](https://github.com/monarch-initiative/mondo-ingest/issues/439)
+- [PR: Add pipeline for externally managed content](https://github.com/monarch-initiative/mondo-ingest/pull/440)

mkdocs.yaml

@@ -31,6 +31,9 @@ nav:
       - NCIT: sources/ncit.md
       - OMIM: sources/omim.md
       - ORDO: sources/ordo.md
+    - Externally managed content:
+      - Overview: externally-managed-content.md
+      - NORD: external/nord.md
     - Mondo source metrics:
       - Overview: metrics.md
       - DO: metrics/doid.md

src/ontology/config/external-content-robot-headers.json

@@ -0,0 +1,11 @@
+{
+    "mondo_id": "ID",
+    "report_ref": "A oboInOwl:hasDbXref",
+    "report_ref_source": ">A oboInOwl:source",	
+    "preferred_name": "A oboInOwl:hasExactSynonym",
+    "preferred_name_source": ">A oboInOwl:hasDbXref",	
+    "synonym_type": ">A oboInOwl:hasSynonymType",	
+    "subset": "AI oboInOwl:inSubset",	
+    "subset_source": ">A oboInOwl:source",	
+    "subset_source2": ">A oboInOwl:source"
+}