Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

add callout for net new snapshots #6521

Open
wants to merge 35 commits into
base: current
Choose a base branch
from
Open

Conversation

mirnawong1
Copy link
Contributor

@mirnawong1 mirnawong1 commented Nov 21, 2024

Adding a callout to snapshot config page to clarify the new yaml config for net new snapshots. exisitng config snapshots can continue using config block, however added guidance on what to do if they want to migrate to new yaml spec.

Resolves #6374


🚀 Deployment available! Here are the direct links to the updated files:

@mirnawong1 mirnawong1 requested a review from a team as a code owner November 21, 2024 22:13
Copy link

vercel bot commented Nov 21, 2024

The latest updates on your projects. Learn more about Vercel for Git ↗︎

Name Status Preview Updated (UTC)
docs-getdbt-com ✅ Ready (Inspect) Visit Preview Nov 29, 2024 0:02am

@github-actions github-actions bot added content Improvements or additions to content size: small This change will take 1 to 2 days to address Docs team Authored by the Docs team @dbt Labs labels Nov 21, 2024
Copy link
Collaborator

@graciegoheen graciegoheen left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the original issue Doug opened was about new snapshots configs (like dbt_valid_to_current and snapshot_meta_column_names) not the "new YAML" spec - can we adjust this PR accordingly?

@mirnawong1
Copy link
Contributor Author

I think the original issue Doug opened was about new snapshots configs (like dbt_valid_to_current and snapshot_meta_column_names) not the "new YAML" spec

thanks @graciegoheen , i did meet with @dbeatty10 last week and thought he meant the new yaml spec. will loop Doug to confirm to make sure before i make further changes.

@@ -150,11 +150,27 @@ Configurations can be applied to snapshots using the [YAML syntax](/docs/build/s

</Tabs>

### Snapshot configuration migration

The latest snapshot YAML configurations introduced in dbt v1.9 (such as [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) and [`dbt_valid_to_current`](/reference/resource-configs/dbt_valid_to_current)) are best suited for new snapshots. For existing snapshots, we recommend the following to avoid any inconsistencies in your snapshots:
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Suggested change
The latest snapshot YAML configurations introduced in dbt v1.9 (such as [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) and [`dbt_valid_to_current`](/reference/resource-configs/dbt_valid_to_current)) are best suited for new snapshots. For existing snapshots, we recommend the following to avoid any inconsistencies in your snapshots:
The latest snapshot configurations introduced in dbt v1.9 (such as [`snapshot_meta_column_names`](/reference/resource-configs/snapshot_meta_column_names) and [`dbt_valid_to_current`](/reference/resource-configs/dbt_valid_to_current)) are best suited for new snapshots. For existing snapshots, we recommend the following to avoid any inconsistencies in your snapshots:

@@ -0,0 +1 @@
Define snapshots using the latest [YAML configurations](/docs/build/snapshots#configuring-snapshots), available in Versionless dbt Cloud and Core v1.9 and higher, in the `snapshot` directory file.
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

is this about the YAML syntax?

I think we're getting two separate things muddied:

  • starting in 1.9 you can define snapshots in yaml
  • starting in 1.9 there are a handful of new configs that you can use, we recommend only using those configs on new snapshots or if you use them on old snapshots you will need to migrate

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ah ok thank you for clarifying! resovled this by having the below for snapshot specific configs and general configs:

snapshot specific configs

Starting from Versionless and Core v1.9, defining snapshots in a .sql file using a config block is a legacy method. You can define snapshots in YAML format using the latest snapshot-specific configurations. For new snapshots, we recommend using these latest configs. If applying them to existing snapshots, you'll need to migrate over.

general configs

Starting from Versionless and Core v1.9, defining snapshots in a .sql file using a config block is a legacy method. You can define snapshots in YAML format using the latest configurations. You can also continue using the legacy method for existing snapshots, but we recommend migrating over to the latest YAML format for better readability and maintainability.

@mirnawong1
Copy link
Contributor Author

hey @graciegoheen - made some add'l tweaks and ready for your review when you can!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
content Improvements or additions to content Docs team Authored by the Docs team @dbt Labs size: small This change will take 1 to 2 days to address
Projects
None yet
Development

Successfully merging this pull request may close these issues.

[Core] - Recommend using new snapshot configurations for net-new snapshots only
4 participants