From a4466c2aa1da759318644b770e634f0edb12335a Mon Sep 17 00:00:00 2001 From: Ly Nguyen Date: Mon, 6 Nov 2023 16:04:46 -0800 Subject: [PATCH 1/9] Release note for job notification updates --- .../02-Nov-2023/job-notifications-rn.md | 15 +++++++++++++++ 1 file changed, 15 insertions(+) create mode 100644 website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md new file mode 100644 index 00000000000..8da60e5d24d --- /dev/null +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md @@ -0,0 +1,15 @@ +--- +title: "Enhancement: Email and Slack job notifications" +description: "November 2023: New quality-of-life improvements for setting up and administering email and Slack job notifications" +sidebar_label: "Enhancement: Job notifications" +sidebar_position: 10 +tags: [Nov-2023] +--- + +There are new quality-of-life improvements in dbt Cloud for email and Slack notifications about your jobs: + +- You can add external email addresses and receive job notifications from them. External emails can be addresses that are outside of your dbt Cloud account and also for configuring notifications in third-party services like Microsoft Teams and PagerDuty. +- You can configure notifications for multiple Slack channels. Previously, you could only configure one Slack channel. +- Any account admin can now edit slack notifications, not just the person who created it. + +To learn more, check out [Job notifications](/docs/deploy/job-notifications). \ No newline at end of file From bb20a8361db93f8e28d6083e3b7b9854743e6a81 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Nov 2023 11:51:04 +0000 Subject: [PATCH 2/9] Update metricflow-commands.md clarify mf command examples avail in cloud --- website/docs/docs/build/metricflow-commands.md | 10 ++++------ 1 file changed, 4 insertions(+), 6 deletions(-) diff --git a/website/docs/docs/build/metricflow-commands.md b/website/docs/docs/build/metricflow-commands.md index 2386dab4ba2..fd120591900 100644 --- a/website/docs/docs/build/metricflow-commands.md +++ b/website/docs/docs/build/metricflow-commands.md @@ -180,9 +180,9 @@ Options: ### Validate-configs This command performs validations against the defined semantic model configurations: + ```bash -dbt sl validate-configs # In dbt Cloud mf validate-configs # In dbt Core @@ -206,20 +206,18 @@ Options: ### Health checks This command performs a health check against the data platform you provided in the configs: + ```bash -dbt sl health-checks #in dbt Cloud - mf health-checks #in dbt Core ``` ### Tutorial Follow the dedicated MetricFlow tutorial to help you get started: + ```bash -dbt sl tutorial # In dbt Cloud - mf tutorial # In dbt Core ``` @@ -522,7 +520,7 @@ mf query --metrics revenue --group-by metric_time__month # In dbt Core To add a dimension filter to a where filter, you have to indicate that the filter item is part of your model and use a template wrapper: {{Dimension('primary_entity__dimension_name')}}. -Here's an example query: dbt sl query --metrics order_total --group-by metric_time --where "{{Dimension('order_id__is_food_order')}} = True".

Before using the template wrapper, however, you will need to set up your terminal to escape curly braces for the filter template to work. +Here's an example query: dbt sl query --metrics order_total --group-by metric_time --where "{{Dimension('order_id__is_food_order')}} = True".

Before using the template wrapper, however, set up your terminal to escape curly braces for the filter template to work.
How to set up your terminal to escape curly braces? From 83f417af71a392e9fa9baf421b099b16238c8051 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Thu, 9 Nov 2023 13:49:08 +0000 Subject: [PATCH 3/9] Update version.md fixing broken link Resolves #4376 --- website/docs/reference/project-configs/version.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/website/docs/reference/project-configs/version.md b/website/docs/reference/project-configs/version.md index 1c947412fcd..7ad6fa7b7d4 100644 --- a/website/docs/reference/project-configs/version.md +++ b/website/docs/reference/project-configs/version.md @@ -31,7 +31,7 @@ A version tag in a `.yml` property file provides the control tag, which informs Starting from version 1.5, dbt will no longer require this configuration in your resource `.yml` files. If you want to know more about why this tag was previously required, you can refer to the [property file FAQs](reference/configs-and-properties#faqs). -For more on property files, see their general [documentation](reference/configs-and-properties#where-can-i-define-properties) on the same page. +For more on property files, see their general [documentation](/reference/configs-and-properties#where-can-i-define-properties) on the same page. From 794d2bcdd626991e0a6836f414d28de3172724bc Mon Sep 17 00:00:00 2001 From: Ly Nguyen Date: Thu, 9 Nov 2023 09:50:52 -0800 Subject: [PATCH 4/9] Feedback --- .../release-notes/02-Nov-2023/job-notifications-rn.md | 6 ++++-- 1 file changed, 4 insertions(+), 2 deletions(-) diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md index 8da60e5d24d..32189c66a8e 100644 --- a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md @@ -8,8 +8,10 @@ tags: [Nov-2023] There are new quality-of-life improvements in dbt Cloud for email and Slack notifications about your jobs: -- You can add external email addresses and receive job notifications from them. External emails can be addresses that are outside of your dbt Cloud account and also for configuring notifications in third-party services like Microsoft Teams and PagerDuty. +- You can add external email addresses and receive job notifications from them. External emails can be: + - Addresses that are outside of your dbt Cloud account + - Third-party integration addresses for configuring notifications to services like Microsoft Teams or PagerDuty - You can configure notifications for multiple Slack channels. Previously, you could only configure one Slack channel. -- Any account admin can now edit slack notifications, not just the person who created it. +- Any account admin can now edit slack notifications, not just the person who created them. To learn more, check out [Job notifications](/docs/deploy/job-notifications). \ No newline at end of file From 9fc32238cb841299fcaac768fe26608ba795d46a Mon Sep 17 00:00:00 2001 From: Ly Nguyen <107218380+nghi-ly@users.noreply.github.com> Date: Thu, 9 Nov 2023 15:13:54 -0800 Subject: [PATCH 5/9] Update website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md --- .../release-notes/02-Nov-2023/job-notifications-rn.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md index 32189c66a8e..660129513d7 100644 --- a/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md +++ b/website/docs/docs/dbt-versions/release-notes/02-Nov-2023/job-notifications-rn.md @@ -8,7 +8,7 @@ tags: [Nov-2023] There are new quality-of-life improvements in dbt Cloud for email and Slack notifications about your jobs: -- You can add external email addresses and receive job notifications from them. External emails can be: +- You can add external email addresses and send job notifications to them. External emails can be: - Addresses that are outside of your dbt Cloud account - Third-party integration addresses for configuring notifications to services like Microsoft Teams or PagerDuty - You can configure notifications for multiple Slack channels. Previously, you could only configure one Slack channel. From 3c01cbeb76c8f1cf26a571ce44fab6810e33fff5 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 10 Nov 2023 09:59:48 +0000 Subject: [PATCH 6/9] Update version.md consolidate --- .../docs/reference/project-configs/version.md | 45 +++---------------- 1 file changed, 6 insertions(+), 39 deletions(-) diff --git a/website/docs/reference/project-configs/version.md b/website/docs/reference/project-configs/version.md index 7ad6fa7b7d4..aa4ef205e7a 100644 --- a/website/docs/reference/project-configs/version.md +++ b/website/docs/reference/project-configs/version.md @@ -8,15 +8,15 @@ import VersionsCallout from '/snippets/_version-callout.md'; - -dbt projects have two distinct types of the `version` tags. This field has a different meaning depending on its location. +dbt projects have two distinct types of `version` tags. This field has a different meaning depending on its location. ## `dbt_project.yml` versions -The version tag in a `dbt_project` file represents the version of your dbt project. Starting in version 1.5, `version` in the `dbt_project.yml` is an *optional parameter*. If specified, the version must be in a [semantic version](https://semver.org/) format, e.g. `1.0.0`. The default value if not specified is `None`. +The version tag in a `dbt_project` file represents the version of your dbt project. Starting in dbt version 1.5, `version` in the `dbt_project.yml` is an *optional parameter*. If used, the version must be in a [semantic version](https://semver.org/) format, such as `1.0.0`. The default value is `None` if not specified. For users on dbt version 1.4 or lower, this tag is required, though it isn't currently used meaningfully by dbt. For more on Core versions, see [About dbt Core versions](/docs/dbt-versions/core). + ```yml @@ -29,7 +29,7 @@ version: version A version tag in a `.yml` property file provides the control tag, which informs how dbt processes property files. -Starting from version 1.5, dbt will no longer require this configuration in your resource `.yml` files. If you want to know more about why this tag was previously required, you can refer to the [property file FAQs](reference/configs-and-properties#faqs). +Starting from version 1.5, dbt will no longer require this configuration in your resource `.yml` files. If you want to know more about why this tag was previously required, you can refer to the [FAQs](#faqs). For users on dbt version 1.4 or lower, this tag is required, For more on property files, see their general [documentation](/reference/configs-and-properties#where-can-i-define-properties) on the same page. @@ -72,39 +72,6 @@ models: +## FAQS - - - - -dbt projects have two distinct types of `version` tags. This field has a different meaning depending on its location. - -## `dbt_project.yml` versions - -The version tag in a `dbt_project` file represents the version of your dbt project and **is a required parameter**. However, it isn't currently used in a meaningful way by dbt. The version must follow a [semantic version](https://semver.org/) format, such as 1.0.0. For more information about dbt Core versions, refer to [About dbt Core versions](/docs/dbt-versions/core). - - -```yml -version: version -``` - - - -## `.yml` property file versions - -A version tag in a `.yml` property file provides the control tag, which informs how dbt processes property files. For more on why we require this tag, see property file [FAQs](reference/configs-and-properties#faqs). - -For more on property files, see their general [documentation](/reference/configs-and-properties#where-can-i-define-properties) on the same page. - - - -```yml -version: 2 # Only 2 is accepted by current and recent versions of dbt. - -models: - ... -``` - - - - + From 406fc5d919cdf8d8dc11d6431ce53962d09296f3 Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 10 Nov 2023 10:06:33 +0000 Subject: [PATCH 7/9] Update version.md --- website/docs/reference/project-configs/version.md | 4 +++- 1 file changed, 3 insertions(+), 1 deletion(-) diff --git a/website/docs/reference/project-configs/version.md b/website/docs/reference/project-configs/version.md index aa4ef205e7a..890ad8542a7 100644 --- a/website/docs/reference/project-configs/version.md +++ b/website/docs/reference/project-configs/version.md @@ -13,7 +13,9 @@ dbt projects have two distinct types of `version` tags. This field has a differe ## `dbt_project.yml` versions -The version tag in a `dbt_project` file represents the version of your dbt project. Starting in dbt version 1.5, `version` in the `dbt_project.yml` is an *optional parameter*. If used, the version must be in a [semantic version](https://semver.org/) format, such as `1.0.0`. The default value is `None` if not specified. For users on dbt version 1.4 or lower, this tag is required, though it isn't currently used meaningfully by dbt. +The version tag in a `dbt_project` file represents the version of your dbt project. + +Starting in dbt version 1.5, `version` in the `dbt_project.yml` is an *optional parameter*. If used, the version must be in a [semantic version](https://semver.org/) format, such as `1.0.0`. The default value is `None` if not specified. For users on dbt version 1.4 or lower, this tag is required, though it isn't currently used meaningfully by dbt. For more on Core versions, see [About dbt Core versions](/docs/dbt-versions/core). From 33c3a5f37454770e7a4625e257d70eba00d47c3c Mon Sep 17 00:00:00 2001 From: mirnawong1 <89008547+mirnawong1@users.noreply.github.com> Date: Fri, 10 Nov 2023 10:13:57 +0000 Subject: [PATCH 8/9] Update semantic-layer-4-build-metrics.md clarify per [slack convo](https://getdbt.slack.com/archives/C05M77P54FL/p1699556859073509?thread_ts=1699525055.794449&cid=C05M77P54FL) --- .../semantic-layer-4-build-metrics.md | 11 +++++++++-- 1 file changed, 9 insertions(+), 2 deletions(-) diff --git a/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-4-build-metrics.md b/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-4-build-metrics.md index cd0efdc9e64..8b112f2f828 100644 --- a/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-4-build-metrics.md +++ b/website/docs/guides/best-practices/how-we-build-our-metrics/semantic-layer-4-build-metrics.md @@ -34,8 +34,15 @@ metrics: ## Query your metric -- It's best practice any time we're updating our semantic layer code to run a `dbt parse && mf validate-configs`. +Use [MetricFlow commands](/docs/build/metricflow-commands#metricflow) for metric validation or queries during development, and apply the following conventions based on your environment: + +- For dbt Cloud, use the `dbt sl` prefix before the command (such as, `dbt sl parse` or `dbt sl query`). +- For dbt Core, use the `mf` prefix (such as `mf validate-configs` or `mf query)`. + +Follow these best practices when updating your semantic layer code, using the `mf` command as an example (replace `mf` with `dbt sl` if you're using dbt Cloud): + +- It's best practice any time we're updating our semantic layer code to run `dbt parse` if using dbt Cloud or `dbt parse && mf validate-configs` if using dbt Core, to validate your configs. - If everything passes, we can start querying this metric with `mf query`! - `mf query` is not how you would use the tool in production, that's handled by the dbt Cloud Semantic Layer's features. It's available for testing results of various metric queries in development, exactly as we're using it now. - Try `mf query --metrics revenue --group-by metric_time__day` and see a preview of the data come back. -- Note the structure of the above query. We select the metric(s) we want and the dimensions to group them by — we use dunders (double underscores e.g.`metric_time__[time bucket]`) to designate time dimensions or other non-unique dimensions that need a specified entity path to resolve (e.g. if you have a orders location dimension and a employee location dimension both named 'location' you would need dunders to specify `orders__location` or `employee__location`). +- Note the structure of the above query. We select the metric(s) we want and the dimensions to group them by — we use dunders (double underscores e.g.`metric_time__[time bucket]`) to designate time dimensions or other non-unique dimensions that need a specified entity path to resolve (e.g. if you have an orders location dimension and an employee location dimension both named 'location' you would need dunders to specify `orders__location` or `employee__location`). From 926c8cec9734b3429d2d068551cc57dc89c3f814 Mon Sep 17 00:00:00 2001 From: Doug Beatty <44704949+dbeatty10@users.noreply.github.com> Date: Fri, 10 Nov 2023 14:54:01 -0700 Subject: [PATCH 9/9] Small formatting changes for configs & properties (#4438) ### Previews - [configs-and-properties](https://docs-getdbt-com-git-dbeatty-config-properties-f-871117-dbt-labs.vercel.app/reference/configs-and-properties) - [dbt_project.yml](https://docs-getdbt-com-git-dbeatty-config-properties-f-871117-dbt-labs.vercel.app/reference/dbt_project.yml) - [docs](https://docs-getdbt-com-git-dbeatty-config-properties-f-871117-dbt-labs.vercel.app/reference/resource-configs/docs) - [source-properties](https://docs-getdbt-com-git-dbeatty-config-properties-f-871117-dbt-labs.vercel.app/reference/source-properties) ## What are you changing in this pull request and why? These are small formatting these found while working on #3989. Merging these changes independently will reduce the diffs in #3989 and make easier to review some of the tricky technical content in that PR. ## Checklist - [x] Review the [Content style guide](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/content-style-guide.md) and [About versioning](https://github.com/dbt-labs/docs.getdbt.com/blob/current/contributing/single-sourcing-content.md#adding-a-new-version) so my content adheres to these guidelines. - [x] I have examined each preview to make sure it looks correct --- website/docs/reference/configs-and-properties.md | 9 ++++++--- website/docs/reference/dbt_project.yml.md | 3 ++- website/docs/reference/resource-configs/docs.md | 10 ++-------- website/docs/reference/source-properties.md | 2 +- 4 files changed, 11 insertions(+), 13 deletions(-) diff --git a/website/docs/reference/configs-and-properties.md b/website/docs/reference/configs-and-properties.md index c2ad5b77629..8a557c762ed 100644 --- a/website/docs/reference/configs-and-properties.md +++ b/website/docs/reference/configs-and-properties.md @@ -11,7 +11,7 @@ A rule of thumb: properties declare things _about_ your project resources; confi For example, you can use resource **properties** to: * Describe models, snapshots, seed files, and their columns -- Assert "truths" about a model, in the form of [tests](/docs/build/tests), e.g. "this `id` column is unique" +* Assert "truths" about a model, in the form of [tests](/docs/build/tests), e.g. "this `id` column is unique" * Define pointers to existing tables that contain raw data, in the form of [sources](/docs/build/sources), and assert the expected "freshness" of this raw data * Define official downstream uses of your data models, in the form of [exposures](/docs/build/exposures) @@ -35,11 +35,11 @@ dbt prioritizes configurations in order of specificity, from most specificity to Note - Generic tests work a little differently when it comes to specificity. See [test configs](/reference/test-configs). -Within the project file, configurations are also applied hierarchically. The most-specific config always "wins": In the project file, configurations applied to a `marketing` subdirectory will take precedence over configurations applied to the entire `jaffle_shop` project. To apply a configuration to a model, or directory of models, define the resource path as nested dictionary keys. +Within the project file, configurations are also applied hierarchically. The most specific config always "wins": In the project file, configurations applied to a `marketing` subdirectory will take precedence over configurations applied to the entire `jaffle_shop` project. To apply a configuration to a model, or directory of models, define the resource path as nested dictionary keys. ### Combining configs -Most configurations are "clobbered" when applied hierarchically. Whenever a more-specific value is available, it will completely replace the less-specific value. Note that a few configs have different merge behavior: +Most configurations are "clobbered" when applied hierarchically. Whenever a more specific value is available, it will completely replace the less specific value. Note that a few configs have different merge behavior: - [`tags`](tags) are additive. If a model has some tags configured in `dbt_project.yml`, and more tags applied in its `.sql` file, the final set of tags will include all of them. - [`meta`](/reference/resource-configs/meta) dictionaries are merged (a more specific key-value pair replaces a less specific value with the same key) - [`pre-hook` and `post-hook`](/reference/resource-configs/pre-hook-post-hook) are also additive. @@ -67,12 +67,14 @@ Previous versions of the docs referred to these as `schema.yml` files — we've dbt has the ability to define node configs in `.yml` files, in addition to `config()` blocks and `dbt_project.yml`. But the reverse isn't always true: there are some things in `.yml` files that can _only_ be defined there. Certain properties are special, because: + - They have a unique Jinja rendering context - They create new project resources - They don't make sense as hierarchical configuration - They're older properties that haven't yet been redefined as configs These properties are: + - [`description`](/reference/resource-properties/description) - [`tests`](/reference/resource-properties/tests) - [`docs`](/reference/resource-configs/docs) @@ -202,3 +204,4 @@ Runtime Error ``` This error occurred because a semicolon (`;`) was accidentally used instead of a colon (`:`) after the `description` field. To resolve issues like this, find the `.yml` file referenced in the error message and fix any syntax errors present in the file. There are online YAML validators that can be helpful here, but please be mindful of submitting sensitive information to third-party applications! + diff --git a/website/docs/reference/dbt_project.yml.md b/website/docs/reference/dbt_project.yml.md index 9bd85d0d5dd..caf501c27ab 100644 --- a/website/docs/reference/dbt_project.yml.md +++ b/website/docs/reference/dbt_project.yml.md @@ -1,5 +1,5 @@ -Every [dbt project](/docs/build/projects) needs a `dbt_project.yml` file — this is how dbt knows a directory is a dbt project. It also contains important information that tells dbt how to operate on your project. +Every [dbt project](/docs/build/projects) needs a `dbt_project.yml` file — this is how dbt knows a directory is a dbt project. It also contains important information that tells dbt how to operate your project. @@ -96,6 +96,7 @@ vars: + ```yml diff --git a/website/docs/reference/resource-configs/docs.md b/website/docs/reference/resource-configs/docs.md index 0ccd21d7504..d300979a826 100644 --- a/website/docs/reference/resource-configs/docs.md +++ b/website/docs/reference/resource-configs/docs.md @@ -17,10 +17,12 @@ default_value: {show: true} { label: 'Macros', value: 'macros', }, ] }> + + ```yml version: 2 @@ -29,7 +31,6 @@ models: docs: show: true | false node_color: "black" - ``` @@ -53,9 +54,7 @@ seeds: - name: seed_name docs: show: true | false - ``` - @@ -71,9 +70,7 @@ snapshots: - name: snapshot_name docs: show: true | false - ``` - @@ -90,7 +87,6 @@ analyses: docs: show: true | false ``` - @@ -110,9 +106,7 @@ macros: - name: macro_name docs: show: true | false - ``` - Also refer to [macro properties](/reference/macro-properties). diff --git a/website/docs/reference/source-properties.md b/website/docs/reference/source-properties.md index d20ef5f2877..1fc7a4aef55 100644 --- a/website/docs/reference/source-properties.md +++ b/website/docs/reference/source-properties.md @@ -1,5 +1,5 @@ --- -title: "About source properties" +title: "Source properties" description: "Learn how to use source properties in dbt." ---