Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Allow storing snapshots outside of the session directory #272

Open
jorisdral opened this issue Jul 3, 2024 · 1 comment
Open

Allow storing snapshots outside of the session directory #272

jorisdral opened this issue Jul 3, 2024 · 1 comment

Comments

@jorisdral
Copy link
Collaborator

jorisdral commented Jul 3, 2024

This means that the snapshots can be stored on a different drive that is more suitable for persisting table snapshots.

The use case is a cloud instance where the local SSD should be considered ephemeral. Active tables can exist on the SSD, but persistent snapshots should live on a persistent network EBS volume.

@dcoutts
Copy link
Collaborator

dcoutts commented Jul 3, 2024

With this feature, it allows multiple sessions to share the same snapshots dir, and thus be used from different processes that do not coordinate on the session or lock file. Thus, when implementing this, we should review the code to create, and delete snapshots to make sure it's safe to use from concurrent sessions.

This probably just means making sure that for creating snapshots we guarantee that the files are new (not existing), so any race results in the loosing side getting an exception. This can be done using the MustBeNew modifier to the OpenMode for the fs-api's hOpen.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants