Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Core] - Recommend using new snapshot configurations for net-new snapshots only #6374

Open
1 task done
dbeatty10 opened this issue Oct 28, 2024 · 3 comments · May be fixed by #6521
Open
1 task done

[Core] - Recommend using new snapshot configurations for net-new snapshots only #6374

dbeatty10 opened this issue Oct 28, 2024 · 3 comments · May be fixed by #6521
Assignees
Labels
content Improvements or additions to content dbt Core The changes proposed in this issue relate to dbt Core dbt-core v1.9 improvement Use this when an area of the docs needs improvement as it's currently unclear

Comments

@dbeatty10
Copy link
Contributor

Contributions

  • I have read the contribution docs, and understand what's expected of me.

Link to the page on docs.getdbt.com requiring updates

https://docs.getdbt.com/reference/snapshot-configs

What part(s) of the page would you like to see updated?

Snapshots and incremental models are meant to be long-lived tables. In particular, snapshots capture history of data that might not be available anymore (or is expensive / difficult to replay). So they are more sensitive to either user bugs in modeling logic or bugs in the underlying dbt software -- in the worst case, there can be unrecoverable data loss.

So to reduce the blast radius of potential bugs in the dbt implementation, in we are recommending using the new v1.9 snapshot configurations with net new snapshots rather than pre-existing ones. This is in the same spirit as dbt-labs/dbt-core#10923 (which might not be possible or reasonable to implement).

Additional information

No response

@dbeatty10 dbeatty10 added content Improvements or additions to content improvement Use this when an area of the docs needs improvement as it's currently unclear dbt-core v1.9 labels Oct 28, 2024
@graciegoheen
Copy link
Collaborator

For new-new snapshots or migrate your old snapshot before running

@hope-wat
Copy link
Contributor

hope-wat commented Oct 29, 2024

or migrate your old snapshot before running

Would we essentially recommend running alter statements manually?

@dbeatty10
Copy link
Contributor Author

Would we essentially recommend running alter statements manually?

Yes. To perform the applicable migration, we'd be recommending first making a backup copy (or clone) and then applying the applicable alter statements either manually or with a script.

@mirnawong1 mirnawong1 self-assigned this Nov 14, 2024
@mirnawong1 mirnawong1 added the dbt Core The changes proposed in this issue relate to dbt Core label Nov 14, 2024
@mirnawong1 mirnawong1 changed the title Recommend using new snapshot configurations for net-new snapshots only Core - Recommend using new snapshot configurations for net-new snapshots only Nov 14, 2024
@mirnawong1 mirnawong1 changed the title Core - Recommend using new snapshot configurations for net-new snapshots only [Core] - Recommend using new snapshot configurations for net-new snapshots only Nov 14, 2024
@mirnawong1 mirnawong1 linked a pull request Nov 21, 2024 that will close this issue
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content dbt Core The changes proposed in this issue relate to dbt Core dbt-core v1.9 improvement Use this when an area of the docs needs improvement as it's currently unclear
Projects
None yet
Development

Successfully merging a pull request may close this issue.

4 participants