[Regression] Jinja templating not working in dependencies.yml

[H-Max]

Is this a regression in a recent version of dbt-core?

• I believe this is a regression in dbt-core functionality
• I have searched the existing issues, and I could not find an existing issue for this regression

Current Behavior

packages.yml file supports jinja templating, at least to read environment variables with env_var().

For instance:

packages:
  - package: dbt-labs/dbt_utils
    version: '{{ env_var("DBT_UTILS_VERSION", "1.3.0") }}'

This does not work anymore when renaming the file to dependencies.yml and this will be an issue in the future.

We use this to pull private dbt packages from gitlab repositories using a syntax like that:

packages:
  - package: dbt-labs/dbt_utils
    version: 1.3.0
  - git: "https://{{ ('gitlab-ci-token:' ~ env_var('CI_JOB_TOKEN') ~ '@') if env_var('CI_JOB_TOKEN', '') }}gitlab.com/path/to/my/package"
    revision: 1.1.1

Expected/Previous Behavior

Jinja templating should be re-enabled in the file dependencies.yml

Steps To Reproduce

Simply add jinja templating trying to read an existing (or not) environment variable in dependencies.yml and you'll get a syntax error.

Relevant log output

No response

Environment

- OS: MacOS
- Python: 3.11.9
- dbt (working version): whatever the version if you're using packages.yml
- dbt (regression version): whatever the version if you're switching to dependencies.yml

Which database adapter are you using with dbt?

bigquery

Additional Context

No response

[dbeatty10]

Hey @H-Max ! I see what you are saying about Jinja templating not being supported in dependencies.yml and how this affects using environment variables for private packages within dependencies.yml.

The solution for your case:

1. Keep any package dependencies within your existing packages.yml file
2. Add any project dependencies to a brand-new dependencies.yml file.

Relevant documentation

We have this documented here, but it is easy to miss:

Next steps

We intentionally chose for dependencies.yml to not include Jinja rendering, so I'm going to close this as not a bug or a regression.

Rather, you can keep any dbt packages that need Jinja rendering within your packages.yml file and use it alongside a new dependencies.yml file.

[H-Max]

Thanks Doug for the heads up to the documentation. I did miss it.

I really thought I read somewhere that it was a good practice to move everything to dependencies.yml, and I wonder if there wasn't a time when there was a warning about packages.yml depreciation, but I am may be wrong about that.

Thanks again for the quick reply on the issue !

[dbeatty10]

I really thought I read somewhere that it was a good practice to move everything to dependencies.yml, and I wonder if there wasn't a time when there was a warning about packages.yml depreciation, but I am may be wrong about that.

You have a good memory @H-Max 🧠

Our original guidance was to move everything to dependencies.yml. The problem is that not everyone can do that. So we updated our guidance in dbt-labs/docs.getdbt.com#6129 / dbt-labs/docs.getdbt.com#6171.

[H-Max]

You have a good memory @H-Max 🧠

I knew it ! I'm not mad ! :D

Well, as long as you don't plan on removing jinja / env_var support in packages.yml, I'm fine with it !