Provide a MassBank schema.org DataFeed

[sneumann]

Hi,
the 2022 Biohackathon has project 23 to consume schema.org DataFeeds. @albangaignard or @AlasdairGray could point us to what's needed to provide our existing schema markup in such a form.
Yours, Steffen

[sneumann]

There is more information in BioSchemas/specifications#579
Yours, Steffen

[sneumann]

We have a RecordExporter command line tool to convert from the MassBank text records to HTML like in the web application:
https://github.com/MassBank/MassBank-web/blob/main/MassBank-Project/MassBank-lib/src/main/scripts/RecordExporter

To create the JSON dump we can either take the exported HTML and extract the <script type="application/ld+json">, or @meier-rene even adds a command-line switch --json-only to export only that.

Yours, Steffen

[sneumann]

I love command line tools:

MassBank-web/MassBank-Project/MassBank-lib/target/MassBank-lib/MassBank-lib/bin/Inspector  \
  IPB_Halle/MSBNK-IPB_Halle-PB005803.txt /dev/stdout \
  | xmllint -html --xpath 'string(//html/head/script[@type = "application/ld+json"]/text())' - 2> /dev/null

and of course also for the entire MassBank-data (not super-fast, though...)

find . -name "MSBNK-IPB_Halle-PB00048*.txt" -exec sh -c "/vol/massbank/src/MassBank-web/MassBank-Project/MassBank-lib/target/MassBank-lib/MassBank-lib/bin/Inspector  {} /dev/stdout | xmllint -html --xpath 'string(//html/head/script[@type = \"application/ld+json\"]/text())' - 2> /dev/null " \; | jq -s 'add' >DataDump.json

[sneumann]

jq is quite strict about proper JSON. SMILES with stereo chemistry inside JSON can pose a problem:
"smiles": "C1CC2=C(C(=CC=C2)O)OC3=CC=CC(=C3)/C=C\C4=CC(=C(C(=C4)O)O)OC5C=CC1C=C5"
will give parse error: Invalid escape at line ... as already mentioned by Tobias in #316 (comment)

So to massage the output we need
cat DataDump.json | sed -e 's#\\#\\\\#' | jq -s 'add' >DataDump.jsonld
Assuming that the smiles are the only place that has a \.
This might not be necessary after fixing #316

[sneumann]

Thanks to @meier-rene we now have a DadaDump created via

MassBank-web/MassBank-Project/MassBank-lib/target/MassBank-lib/MassBank-lib/bin/Msbnk2JSONLD -o MassBank-2006.06.jsonld $(ls -d MassBank-data/* )

As sample is available from
https://msbi.ipb-halle.de/~sneumann/MassBank-2006.06.jsonld
Yours, Steffen

[sneumann]

Now that we know how to create a Data Feed, we need to serve it.
According to https://schema.org/docs/feeds.html this goes into
/.well-known/feeddata-general with the option to split if it becomes too large.
That data feed should be created upon data import.

[sneumann]

We should also find a way to express the massbank-data version (git sha256 and/or the release) of a DataFeed.

[tsufz]

/.well-known/ is located in the Apache root to maintain the Letsencrypt challenge. The feeddata-general would be created in the Tomee root. Once implemented, we can redirect the request to the Tomee root.

[sneumann]

The DataDump is now created as part of the release process in

MassBank-web/create-releasefiles.sh

Line 13 in 18d97bd

./MassBank-lib/target/MassBank-lib/MassBank-lib/bin/RecordExporter -f jsonld -o MassBank.json ../../MassBank-data/*

and files go to the GitHub releases:
https://github.com/MassBank/MassBank-data/releases/latest/download/MassBank.json
We do not have that file in the running MassBank server yet.
Yours,
Steffen