Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add/generic vector ingests #220

Merged
merged 39 commits into from
Aug 14, 2024
Merged

Add/generic vector ingests #220

merged 39 commits into from
Aug 14, 2024

Conversation

paridhi-parajuli
Copy link
Contributor

Summary: Refactor code to support vector data ingest
Addresses Issue

Changes

  • Generalized code for vector data ingest and removed eis-related functionalies.
  • Change default value for EXTERNAL_ROLE_ARN passed to the vector ingest ECS
  • Add configs for loading data into features database
  • Add function to assume AWS IAM role and return temporary credentials

PR Checklist

  • Update CHANGELOG
  • Unit tests
  • Ad-hoc testing - Deploy changes and test manually
  • Integration tests

@slesaad slesaad requested a review from smohiudd August 14, 2024 16:50
Copy link
Member

@slesaad slesaad left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks good to me. @amarouane-ABDELHAK , @smohiudd , @ranchodeluxe , you guys ok with merging this?

README.md Outdated Show resolved Hide resolved
README.md Outdated Show resolved Hide resolved
@smohiudd
Copy link
Contributor

smohiudd commented Aug 14, 2024

I just added some general comments on documentation. The way the original EIS vector dag is labelled it's hard to know the difference between that and the generic. So in the docs would be good to be specific that the ingest_vector DAG has logic that is tailored to EIS Fire datasets only

@paridhi-parajuli paridhi-parajuli merged commit cee64f5 into dev Aug 14, 2024
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

4 participants