Check against srophe app to ensure record doesn't already exist

[wlpotter]

Use the app index (figure out linking to it)

Specify the app index in config.xml

[wlpotter]

Priority for implementing taxonomy module.

[wlpotter]

@dlschwartz if memory serves you use something in a schematron rule to check to see if a referenced Syriaca URI exists in our data/on the app, is that right? Maybe this is for tei:bibl/tei:ptr/@target values?

Does that sound familiar? I'm wondering if we could use a similar functionality for the transform.

[wlpotter]

@dlschwartz I may have found them: https://github.com/srophe/srophe-eXist-app/tree/master/documentation/indexes

(Cf. srophe/srophe-eXist-app#1239)

We only have persons and places, and I'm not sure how these are generated.

[wlpotter]

Note to self: The xquery module that builds these indices is here: https://github.com/srophe/srophe-eXist-app/blob/master/modules/build-index.xql

Could have this adapted to run on each entity collection?

[dlschwartz]

@wlpotter Winona produced these. Ideally we should have these automated to run periodically. It would be nice if the uploading of new records to the master branch of srophe-app-data would trigger an update. The index for the taxonomy is here: https://github.com/srophe/srophe-app-data/blob/master/data/subjects/taxonomyIndex.xml. It should instead be in the same place as the person and place records.

[dlschwartz]

The xslt is here: https://github.com/srophe/srophe-app-data/blob/master/data/subjects/taxonomyIndex.xsl. I need to modify this as we tweak the data but I would like to keep this xslt. It produces an index that groups the keywords in a range of ways. I use those groupings to validate SPEAR records so I'll want to maintain those.

[dlschwartz]

And yes, I think we should have an index for each data type. Ideally they could all be automated to run whenever we upload one or more new records. If not that, maybe we could have a maintenance script that would just run periodically.

[wlpotter]

For the non-taxonomy entities, the following is needed

• add the index URIs to the config: in /parameters/config.xml add as /meta/config/collections/collection/@index
• update the script to have it check each URI from the doc/csv and if it is in the index already than return an error (as a string or as an XML snippet of an error) which should return to console rather than a file regardless of the fileOrConsole setting
• create a srophe app issue (check that one doesn't exist already) to have the index get rebuilt automatically whenever a GitHub webhook is triggered

I've split off taxonomy to #37

[wlpotter]

I believe I have this working, though I'm leaving open for testing. This also may get tweaked slightly as I work on #27. Currently it just returns an un-namespaced <error> element with a description of the error. I also want to pass an initial warning to the console if there is a problem accessing an index during run-time so the user knows to be careful when merging in new data.

[wlpotter]

I've tested and can confirm that this raises an error to the console letting the use know a given URI already exists (regardless of file/console setting); it also does not create the record if it already exists.

I have also prepended a console message whenever the index is not available (either because it returns 404 or no index is selected). This is just given as a warning so that we can keep overwriting the taxonomy during its current development.

I believe this issue is fixed.