[Epic] Design Data Package Pipelines Trial

[rufuspollock]

We anticipate Data Package Pipelines can be very useful for performing automated tasks such as scraping and merging -- as well as a demonstration of how to build and deploy async worker infrastructure.

However, as yet we have not used Data Package Pipelines and so we need to:

• Research and understand them better
• Design an MVP trial to test them out and deliver some immediate value

DPP Meeting document

Acceptance Criteria

• Set out the technical and business case for Data Package Pipelines
• MVP Plan for usage and integration

Tasks

• Research Data Package Pipelines - what are they, how they work - Understand use of datapackage pipelines #350
• Identify use cases we have where DPP are relevant - Identify use cases we have where DP Pipelines are relevant #357
• Design a trial usage - "MVP" - Design a trial usage of DP Pipelines - "MVP" #358