Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Stateful bulk export/import on AWS #515

Closed
johngrimes opened this issue May 3, 2022 · 1 comment
Closed

Stateful bulk export/import on AWS #515

johngrimes opened this issue May 3, 2022 · 1 comment
Labels
new feature New feature or request

Comments

@johngrimes
Copy link
Member

Currently the import lambdas support the since parameter for bulk exports, but this is configured in and no help is provided with respect to storing and updating this parameter upon each import.

The lambdas could be enhanced to optionally allow for the storage and update of the since parameter using DynamoDB. This would facilitate incremental exports and the loading of only new and changed content. The merge mode of the Pathling import operation could be used to selectively update tables, rather than replacing them.
@johngrimes johngrimes added the new feature New feature or request label May 3, 2022
@johngrimes johngrimes moved this to Todo in Pathling Aug 26, 2022
@johngrimes
Copy link
Member Author

Closed in favour of #1776.

@github-project-automation github-project-automation bot moved this from Backlog to Done in Pathling Jan 17, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
new feature New feature or request
Projects
Pathling
Status: Done
Milestone
No milestone
Development

No branches or pull requests
1 participant
@johngrimes