Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

duplicated output #26

Open
retsys opened this issue May 28, 2020 · 4 comments
Open

duplicated output #26

retsys opened this issue May 28, 2020 · 4 comments

Comments

@retsys
Copy link

retsys commented May 28, 2020

Hello,
I'm running airflow-prometheus-exporter v1.0.8 in airflow v1.10.10
Seems to be a cartesian in the output.
Lines containing "airflow_dag_run_duration{dag_id=..." are repeated multiple times for each dag.
Seems to be one line for each dag run across all dags.

As in, airflow reports a total of y dag runs since it started if you sum up all dag runs from all dags.
Then there are y lines of "airflow_dag_run_duration{dag_id=..." in this output for each dag.
Lots of dags x lots of dag runs = lots of output and growing all the time.
Meaning scrape times are growing too.
Cheers

@qiuwei
Copy link

qiuwei commented Jul 21, 2020

confirm that I also encountered the same issue.

qiuwei added a commit to qiuwei/airflow-prometheus-exporter that referenced this issue Jul 21, 2020
remove dag_duration temporarily. see robinhood#26
@jutley
Copy link

jutley commented Sep 29, 2020

We are running into this as well. When it runs for long enough, the result is that metrics take so long to return that we essentially lose our airflow metrics.

@andreyvital
Copy link

Same issue here!

@jward-bw
Copy link

jward-bw commented Oct 8, 2020

I have a PR raised to fix this, but it seems that the owner of the repo is no longer active:
#25

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

5 participants