How to capture metadata from Dagster

[nehiljain]

It would be amazing if I can centralize metadata of dag runs inside Dagster and lineage from Dbt on snowflake in one repository.

Dagster IO Metadata API

dbt Metadata API

Then I can start doing some analysis in my CICD pipelines to prevent failures from breaking changes between consumers and producers. Eventually enforcing data contracts.

[criccomini]

Going to convert this to a GH issue.

[criccomini]

So it sounds like there are a few things here:

1. Dagster jobs in Recap
2. dbt job lineage in Recap

Does that sound right?

[nehiljain]

Yep, these two would be analyzers for the two.