Develop an Indexing Pipeline

[tpendragon]

This will have a bunch of sub-issues, this is mostly to record the fact that we need to make this.

Acceptance Criteria

• Create an Architecture Decision Record for indexing. #41
• Create a fixture database container that has Figgy resources in it we can test against #42
• Develop the Hydration Phase #43
• Develop the Transformation Phase #44
• Develop the Indexing Phase #45
  • Indexing phase should not kill Solr. Have some way to throttle traffic to Solr.
• Develop a validation strategy to ensure everything that we want indexed got indexed. #46
• Create and document a full rehydration, retransformation, and reindexing process #47

First Step

Split those tickets up up there.

Finalize the ADR: #34

Architecture Discussions

https://docs.google.com/document/d/1VgYeonai-q4NI6yXK0n3rYAipDsHH7j85C3XxoP-W6E/edit#heading=h.8n3o7dn0pcfs

Content

All public Ephemera resources that are images