You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The goal with this Epic is to allow us to track how is the DAG Factory adoption by Astro customers, if possible at a DAG run level.
@cmarteepants has been analysing how we can measure DAG factory adoption. Apparently , as of v0.19.0, DAG factory does not have a way of identifying in the created DAGs that they were created by DAG factory.
What Constance did was to align with multiple teams within Astronomer so we programatically query to find which dependencies are installed in Astro deployments. With this information, we'd know how many Astro customers have DAG factory installed.
As part of this epic, we review this approach and check if / how we should give visibility of which DAGs were created with DAG factory. The solution may only work with future versions of DAG Factory.
Some approaches we could look into:
Generate some events during the conversion of DAG Factory using Add Scarf based telemetry apache/airflow#39510 (would only work for newer versions of Airflow, or we'd need to have DAG Factory using this tool to emit them)
Check the possibility of annotating the DAGs created by DAG Factory in other ways - making sure we can query those in the Astro SF database
Description co-authored by @tatiana @pankajastro
The goal with this Epic is to allow us to track how is the DAG Factory adoption by Astro customers, if possible at a DAG run level.
@cmarteepants has been analysing how we can measure DAG factory adoption. Apparently , as of v0.19.0, DAG factory does not have a way of identifying in the created DAGs that they were created by DAG factory.
What Constance did was to align with multiple teams within Astronomer so we programatically query to find which dependencies are installed in Astro deployments. With this information, we'd know how many Astro customers have DAG factory installed.
As part of this epic, we review this approach and check if / how we should give visibility of which DAGs were created with DAG factory. The solution may only work with future versions of DAG Factory.
Some approaches we could look into:
Tasks associated:
More will likely be created after this PoC
The text was updated successfully, but these errors were encountered: