Design/Prototype for Data Adapter storage

[waab76]

As a Graylog user, I want to take advantage of threat intel streams that are distributed as large data sets without having to keep all of the data in main memory.

Input Criteria

• Basic requirements for threat intel data streams known

Output Criteria

• Technical path forward identified for handling data from bulk-download threat intel streams
• Prototype code written to demonstrate

Tasks

• Design a mechanism for storing and utilizing data sets from bulk download sources (likely MongoDB)
• Discuss design proposal with team (include Bernd)
• Implement proof-of-concept code to show that the approach works