Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] To support task display_name #1277

Open
1 task done
t0momi219 opened this issue Oct 23, 2024 · 0 comments · May be fixed by #1278
Open
1 task done

[Feature] To support task display_name #1277

t0momi219 opened this issue Oct 23, 2024 · 0 comments · May be fixed by #1278
Labels
area:rendering Related to rendering, like Jinja, Airflow tasks, etc enhancement New feature or request triage-needed Items need to be reviewed / assigned to milestone

Comments

@t0momi219
Copy link
Contributor

t0momi219 commented Oct 23, 2024 •
edited
Loading

Description

When running models that have names containing multibyte characters, runtime errors occur in Airflow environments where statsd is enabled (MWAA uses this statsd metric for collecting metrics in Cloudwatch).

Related Issue: apache/airflow#18010

To address this, Airflow 2.9 introduced the ability to render tasks using display_name, which allows task names to be rendered separately from their task_id.

Reference: https://airflow.apache.org/docs/apache-airflow/stable/_modules/airflow/example_dags/example_display_name.html

It would be ideal to have a mechanism that allows users to set a different task_id while using the model name as the display_name when working with models that contain multibyte characters.

Use case/motivation

The following error occurs in MWAA environment:

[2024-10-17, 10:27:03 UTC] {taskinstance.py:2865} INFO - Starting attempt 1 of 1
[2024-10-17, 10:27:03 UTC] {validators.py:135} ERROR - Invalid stat name: dag.run_cosmos_dbt.日本語名モデル_run.queued_duration.
Traceback (most recent call last):
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/metrics/validators.py", line 132, in wrapper
    stat = handler_stat_name_func(stat)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/usr/local/airflow/.local/lib/python3.11/site-packages/airflow/metrics/validators.py", line 220, in stat_name_default_handler
    raise InvalidStatsNameException(
airflow.exceptions.InvalidStatsNameException: The stat name (dag.run_cosmos_dbt.日本語名モデル_run.queued_duration) has to be composed of ASCII alphabets, numbers, or the underscore, dot, or dash characters.

Related issues

No response

Are you willing to submit a PR?

  • Yes, I am willing to submit a PR!
@t0momi219 t0momi219 added enhancement New feature or request triage-needed Items need to be reviewed / assigned to milestone labels Oct 23, 2024
@t0momi219 t0momi219 linked a pull request Oct 23, 2024 that will close this issue
Open
2 tasks
@dosubot dosubot bot added the area:rendering Related to rendering, like Jinja, Airflow tasks, etc label Oct 23, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
area:rendering Related to rendering, like Jinja, Airflow tasks, etc enhancement New feature or request triage-needed Items need to be reviewed / assigned to milestone
Projects
None yet
Milestone
No milestone
Development

Successfully merging a pull request may close this issue.

To support task display_name t0momi219/astronomer-cosmos
1 participant
@t0momi219