Skip to content

Latest commit

 

History

History
60 lines (42 loc) · 2.78 KB

README.md

File metadata and controls

60 lines (42 loc) · 2.78 KB
Raw

Deploy CCC

Dependencies

  • install dependencies with pip install -r requirements.txt
  • In ccc/scripts, put Blazegraph.jar Direct link to download

RUN BEE LOCALLY

  • Change bee/conf.py content with content of bee/conf_local.py
  • Change ccc/conf_spacin.py content with content of ccc/conf_spacin_local.py
  • Shell (from ccc/scripts): python3 -m script.ccc.run_bee. It creates a folder called test in the same folder scripts.
  • OUTPUT JSON: scripts/test/share/ref/todo
  • ERRORS: scripts/test/index/ref/issue

Rerun BEE

  • Empty/remove the folder test/
  • Run: python3 -m script.ccc.run_bee

Run BEE on a single XML file

  • Include the XML file in the folder script/ccc/
  • Uncomment lines 39, 40 of script/ccc/jats2oc.py
  • Change in script/ccc/test_bee.py the name of the file to be parsed
  • Run: python3 -m script.ccc.test_bee

RUN SPACIN LOCALLY

  • INPUT JSON: scripts/test/share/ref/todo
  • OUTPUT RDF (dump): scripts/ccc/
  • Run Blazegraph: java -Dfile.encoding=UTF-8 -Dsun.jnu.encoding=UTF-8 -server -Xmx1g -Djetty.port=9999 -Dbigdata.propertyFile=ccc.properties -jar blazegraph.jar
  • Run: python3 -m script.ccc.run_spacin

Rerun SPACIN

  • Empty scripts/ccc/ BUT do not remove scripts/ccc/context.json
  • Remove scripts/ccc.jnl (quit the .jar first!)
  • If you want to rerun SPACIN on the same JSON files, move the content of scripts/test/share/ref/done into scripts/test/share/ref/todo

Other notes:

  • do not change the config file script/ccc/conf_bee.py
  • do not delete context.json included in scripts/ccc/ when rerunning SPACIN

Exploiting local indexes

BEE and SPACIN have been enhanced in order to exploit respectively a CSV dataset generated with europe-pubmed-central-dataset tool and papendex tool.

  • (BEE) in scripts/script/bee/conf.py there are:

    • PARALLEL_PROCESSING: set to True in order to enable the improvement made
    • dataset_reference: absolute reference to the CSV generated
    • article_path_reference: absolute reference to the directory where all the XML articles are stored
    • n_process: the number of processes that will be spawned.
    • doc_for_process: the CSV will be splitted in a number of chunks (one for each process), having the number of docs specified here

  • (SPACIN) in script/ccc/conf_spacin.py there are:

    • crossref_query_interface_type: set to 'local' if you want to exploit the local index, otherwise 'remote'
    • orcid_query_interface_type = set to 'local' if you want to exploit the local index, otherwise 'remote'