Bump dbt from 0.17.2 to 1.0.0 #105

dependabot · 2021-12-07T08:04:23Z

Bumps dbt from 0.17.2 to 1.0.0.

Release notes

dbt-core v1.0.0

dbt Core 1.0.0 - W. E. B. Du Bois (December 3, 2021)

💬 Discourse: v1.0

📖 New & changed documentation

Breaking changes

The two type of test definitions are now "singular" and "generic" (instead of "data" and "schema", respectively). The test_type: selection method accepts test_type:singular and test_type:generic. (It will also accept test_type:schema and test_type:data for backwards compatibility) (#3234, #3880). Not backwards compatible: The --data and --schema flags to dbt test are no longer supported, and tests no longer have the tags 'data' and 'schema' automatically applied.

Deprecated the use of the packages arg adapter.dispatch in favor of the macro_namespace arg. (#3895)

Enable on-run-start and on-run-end hooks for dbt test. Add flags.WHICH to execution context, representing current task (#3463, #4004)

Replace greedy flag/property for test selection with indirect_selection: eager/cautious flag/property. Set to eager by default. Note: This reverts test selection to its pre-v0.20 behavior by default. dbt test -s my_model will select multi-parent tests, such as relationships, that depend on unselected resources. To achieve the behavior change in v0.20 + v0.21, set --indirect-selection=cautious on the CLI or indirect_selection: cautious in yaml selectors. (#4082, #4104)

In v1.0.0, pip install dbt will raise an explicit error. Instead, please use pip install dbt-<adapter> (to use dbt with that database adapter), or pip install dbt-core (for core functionality). For parity with the previous behavior of pip install dbt, you can use: pip install dbt-core dbt-postgres dbt-redshift dbt-snowflake dbt-bigquery (#4100, #4133)

Reorganize the global_project (macros) into smaller files with clearer names. Remove unused global macros: column_list, column_list_for_create_table, incremental_upsert (#4154)

Introduce structured event interface, and begin conversion of all legacy logging (#3359, #4055)

This is a breaking change for adapter plugins, requiring a very simple migration. See events module README for details.

If you maintain another kind of dbt-core plugin that makes heavy use of legacy logging, and you need time to cut over to the new event interface, you can re-enable the legacy logger via an environment variable shim, DBT_ENABLE_LEGACY_LOGGER=True. Be advised that we will remove this capability in a future version of dbt-core.

Restrict secret env vars (prefixed DBT_ENV_SECRET_) to profiles.yml + packages.yml only. Raise an exception if a secret env var is used elsewhere (#4310, #4311)

Reorder arguments to config.get() so that default is second (#4273, #4297)

Features

Normalize global CLI arguments/flags (#2990, #3839)

Turns on the static parser by default and adds the flag --no-static-parser to disable it. (#3377, #3939)

Generic test FQNs have changed to include the relative path, resource, and column (if applicable) where they are defined. This makes it easier to configure them from the tests block in dbt_project.yml (#3259, #3880

Turn on partial parsing by default (#3867, #3989)

Generic test can now be added under a generic subfolder in the test-paths directory. (#4052)

Add result:<status> selectors to automatically rerun failed tests and erroneous models. This makes it easier to rerun failed dbt jobs with a simple selector flag instead of restarting from the beginning or manually running the dbt models in scope. (#3859, #4017)

dbt init is now interactive, generating profiles.yml when run inside existing project (#3625)

Avoid error when missing column in YAML description (#4151, #4285)

Allow --defer flag to dbt snapshot (#4110, #4296)

Install prerelease packages when version explicitly references a prerelease version, regardless of install-prerelease status (#4243, #4295)

Add data attributes to json log messages (#4301)

Add event codes to all log events (#4319)

Allow nullable error_after in source freshness (#3874, #3955)

Add metrics nodes (#4071, #4235)

Add support for dbt init <project_name>, and support for skip_profile_setup argument (dbt init -s) (#4156, #4249)

Fixes

Add generic tests defined on sources to the manifest once, not twice (#3347, #3880)

Skip partial parsing if certain macros have changed (#3810, #3982)

Enable cataloging of unlogged Postgres tables (3961, #3993)

Fix multiple disabled nodes (#4013, #4018)

Fix multiple partial parsing errors (#3996, #4020)

Return an error instead of a warning when runing with --warn-error and no models are selected (#4006, #4019)

Fixed bug with error_if test option (#4070)

Allow retries when the answer from a dbt deps is None (#4178, #4225)

Allow specifying default in Jinja config.get with default keyword (#4273, #4297)

Fix serialization error with missing quotes in metrics model ref (#4252, #4287)

Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)

Fix serialization error with missing quotes in metrics model ref (#4252, #4287)

Correct definition of 'created_at' in ParsedMetric nodes (#4298, #4299)

... (truncated)

Changelog

Sourced from dbt's changelog.

dbt-core 1.0.0 (December 3, 2021)

Fixes

Configure the CLI logger destination to use stdout instead of stderr (#4368)

Make the size of EVENT_HISTORY configurable, via EVENT_BUFFER_SIZE global config (#4411, #4416)

Change type of log_format in profiles.yml user config to be string, not boolean (#4394)

Under the hood

Only log cache events if LOG_CACHE_EVENTS is enabled, and disable by default. This restores previous behavior (#4369)

Move event codes to be a top-level attribute of JSON-formatted logs, rather than nested in data (#4381)

Fix failing integration test on Windows (#4380)

Clean up warning messages for clean + deps (#4366)

Use RFC3339 timestamps for log messages (#4384)

Different text output for console (info) and file (debug) logs (#4379, #4418)

Remove unused events. More structured ConcurrencyLine. Replace \n message starts/ends with EmptyLine events, and exclude EmptyLine from JSON-formatted output (#4388)

Update events module README (#4395)

Rework approach to JSON serialization for events with non-standard properties (#4396)

Update legacy logger file name to dbt.log.legacy (#4402)

Rollover dbt.log at 10 MB, and keep up to 5 backups, restoring previous behavior (#4405)

Use reference keys instead of full relation objects in cache events (#4410)

Add node_type contextual info to more events (#4378)

Make materialized config optional in node_type (#4417)

Stringify exception in GenericExceptionOnRun to support JSON serialization (#4424)

Add "interop" tests for machine consumption of structured log output (#4327)

Relax version specifier for dbt-extractor to ~=0.4.0, to support compiled wheels for additional architectures when available (#4427)

dbt-core 1.0.0rc3 (November 30, 2021)

Fixes

Support partial parsing of env_vars in metrics (#4253, #4322)

Fix typo in UnparsedSourceDefinition.__post_serialize__ (#3545, #4349)

Under the hood

Change some CompilationExceptions to ParsingExceptions (#4254, #4328)

Reorder logic for static parser sampling to speed up model parsing (#4332)

Use more augmented assignment statements (#4315), (#4311)

Adjust logic when finding approximate matches for models and tests (#3835), #4076)

Restore small previous behaviors for logging: JSON formatting for first few events; WARN-level stdout for list task; include tracking events in dbt.log (#4341)

Contributors:

@sarah-weatherbee (#4331)

@emilieschario (#4076)

@sneznaj (#4349)

dbt-core 1.0.0rc2 (November 22, 2021)

Breaking changes

Restrict secret env vars (prefixed DBT_ENV_SECRET_) to profiles.yml + packages.yml only. Raise an exception if a secret env var is used elsewhere (#4310, #4311)

Reorder arguments to config.get() so that default is second (#4273, #4297)

... (truncated)

Commits

ec1f609 Bumping version to 1.0.0 (#4431) (#4432)
b4ea003 Changelog entries for rc3 -> final (#4389) (#4430)
23e1a9a relax version specifier for dbt-extractor (#4427) (#4429)
9882d08 add new interop tests for black-box json log schema testing (#4327) (#4428)
79cc811 stringify generic exceptions (#4424) (#4425)
c82572f Info vs debug text formatting (#4418) (#4421)
42a38e4 Sources aren't materialized (#4417) (#4420)
ecf0ffe Add flag to main.py. Reinstantiate after flags (#4416) (#4419)
e9f26ef add node type codes to more events + more hook log data (#4378) (#4415)
c77dc59 use reference keys instead of relations (#4410) (#4414)
Additional commits viewable in compare view

Dependabot will resolve any conflicts with this PR as long as you don't alter it yourself. You can also trigger a rebase manually by commenting @dependabot rebase.

Dependabot commands and options

You can trigger Dependabot actions by commenting on this PR:

@dependabot rebase will rebase this PR
@dependabot recreate will recreate this PR, overwriting any edits that have been made to it
@dependabot merge will merge this PR after your CI passes on it
@dependabot squash and merge will squash and merge this PR after your CI passes on it
@dependabot cancel merge will cancel a previously requested merge and block automerging
@dependabot reopen will reopen this PR if it is closed
@dependabot close will close this PR and stop Dependabot recreating it. You can achieve the same result by closing it manually
@dependabot ignore this major version will close this PR and stop Dependabot creating any more for this major version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this minor version will close this PR and stop Dependabot creating any more for this minor version (unless you reopen the PR or upgrade to it yourself)
@dependabot ignore this dependency will close this PR and stop Dependabot creating any more for this dependency (unless you reopen the PR or upgrade to it yourself)