Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Implement tool for distributing a KG as BdBags #9

Closed
cmungall opened this issue Apr 25, 2018 · 3 comments
Closed

Implement tool for distributing a KG as BdBags #9

cmungall opened this issue Apr 25, 2018 · 3 comments

Comments

@cmungall
Copy link
Contributor

The BdBag would contain individual files representing slices of the graph in an agreed upon format (NCATS-Tangerine/translator-knowledge-graph#6). The KG could be slice in any number of ways but a standard way would be by source.

cc @stevencox

@stevencox
Copy link

cool.

@cmungall
Copy link
Contributor Author

cmungall commented May 9, 2018

not sure if @kltm would have any python code we could reuse here

@kltm
Copy link
Member

kltm commented May 9, 2018

Direct full bdbags are easy given the python libs as we use them. Slightly more nuanced is the creation of "holey" bdbags and getting the references and hashes sorted out. Code that may be interesting for that would be:

https://github.com/geneontology/go-site/blob/master/scripts/create-bdbag-remote-file-manifest.py

In use like:

sh 'python3 ./scripts/create-bdbag-remote-file-manifest.py -v --walk $WORKSPACE/mnt/$BRANCH_NAME/ --remote http://release.geneontology.org/$START_DATE --output manifest.json'
sh 'mkdir go-release-reference'
sh 'python3 ./mypyenv/bin/bdbag ./go-release-reference --remote-file-manifest manifest.json --archive tgz'

Ideally, that script may be better as a lib or cli to pypi.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants