Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Closes #1483: Set collectedfrom to OpenAIRE in all the relations exported by the IIS #1484

Open
wants to merge 1 commit into
base: master
Choose a base branch
from

Conversation

marekhorst
Copy link
Member

My initial impression was it is going to be just a simple addition/extension but it turned out I had to modify quite a substantial part of the exporter module. Mostly due to having different Spark exporter classes for different relation types bound to exported entities (patent, software, citation relations) and a whole different stack for plain relations coming from text mining algorithms covered with classes extending AbstractRelationBuilderModule.

I am adding support for export_relation_collectedfrom_value IIS input parameter specifying datasource identifier of all relations exported by the IIS.

So every exported relation is going to have Relation#collectedfrom[0]#key set to predefined OpenAIRE value and Relation#collectedfrom[0]#value set to export_relation_collectedfrom_value input parameter value provided at runtime (already added to default-config.xml file: https://git.icm.edu.pl/openaire/iis-deployment/-/commit/e12d3e4a23517b8e1d5601aa077c0293d8f08d40).

The already existing exporter modules class hierarchy was changed by introducing AbstractRelationBuilderModule to cover collectedfrom related processing when building Relation object.

The existing code is simplified by moving the code responsible for building Relation object to BuilderModuleHelper class. Supplementing unit tests suite with the relevant tests in BuilderModuleHelperTest also covering missing tests for other utiliy methods from the BuilderModuleHelper class.

export_relation_collectedfrom_value was introduced in multiple workflow.xml files starting from IIS primary main workflow and going down to the exporter subworkflow.

Integration tests were extended with additional "expectations" encoded in multiple *.properties files with expected collectedfrom key and value.

…rted by the IIS

Introducing support for `export_relation_collectedfrom_value` input parameter specifying datasource identifier of all relations exported by the IIS.
Extending all exporter modules to include `export_relation_collectedfrom_value` in `Relation#collectedfrom[0]#value` of exported payloads.
@marekhorst marekhorst requested a review from mpol October 22, 2024 14:22
@marekhorst marekhorst self-assigned this Oct 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant