Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add Prometheus Exporter for Exemplars #4178

Open
wants to merge 57 commits into
base: main
Choose a base branch
from

Conversation

czhang771
Copy link

@czhang771 czhang771 commented Sep 4, 2024

Description

Fixes #2407
Follow-up of #4094

  • Add exemplars to histogram in Prometheus exporter
  • Fix typing of span_id and trace_id in exemplar related code (they were set as str when they actually are int)

Note

The Prometheus exporter seems to be implemented differently in Python compared to Java and Dotnet and uses Metric Families and a custom collector rather than directing converting the OTEL types to Prometheus types.
My current attempt focuses on the Histogram and adds the exemplars to each bucket for the histogram although I am not sure if this is the best way to go about it.

Relevant Resources:

Type of change

New feature (non-breaking change which adds functionality)

How Has This Been Tested?

Add a test to check histogram exemplar are exported with the openmetrics format: exporter/opentelemetry-exporter-prometheus/tests/test_prometheus_exporter.py::TestPrometheusMetricReader::test_histogram_with_exemplar_to_prometheus

Does This PR Require a Contrib Repo Change?

  • Yes. - Link to PR:
  • No.

Checklist:

  • Followed the style guidelines of this project
  • Changelogs have been updated
  • Unit tests have been added
  • Documentation has been updated

@fcollonval
Copy link
Contributor

fcollonval commented Oct 16, 2024

@czhang771 would you mind turning this pull request as ready and update the description with:

# Description

Fixes #2407
Follow-up of https://github.com/open-telemetry/opentelemetry-python/pull/4094

- Add exemplars to histogram in Prometheus exporter
- Fix typing of `span_id` and `trace_id` in exemplar related code (they were set as `str` when they actually are `int`)

> [!NOTE]
> The Prometheus exporter seems to be implemented differently in Python compared to Java and Dotnet and uses Metric Families and a custom collector rather than directing converting the OTEL types to Prometheus types.
> My current attempt focuses on the Histogram and adds the exemplars to each bucket for the histogram although I am not sure if this is the best way to go about it.

Relevant Resources:
- Prometheus Metric Protofiles: https://github.com/prometheus/client_model/blob/master/io/prometheus/client/metrics.proto
- Java Prometheus Exporter: https://github.com/open-telemetry/opentelemetry-java/tree/main/exporters/prometheus/src/main/java/io/opentelemetry/exporter/prometheus
- Metric Family Class Types: https://github.com/prometheus/client_python/blob/bed7432fad70a16acff5e7b03a65eb9ec350d3e6/prometheus_client/metrics_core.py#L178
- Custom Collectors in Prometheus: https://prometheus.github.io/client_python/collector/custom/

## Type of change

New feature (non-breaking change which adds functionality)

# How Has This Been Tested?

Add a test to check histogram exemplar are exported with the openmetrics format: `exporter/opentelemetry-exporter-prometheus/tests/test_prometheus_exporter.py::TestPrometheusMetricReader::test_histogram_with_exemplar_to_prometheus`

# Does This PR Require a Contrib Repo Change?

- [ ] Yes. - Link to PR:
- [x] No.

# Checklist:

- [x] Followed the style guidelines of this project
- [x] Changelogs have been updated
- [x] Unit tests have been added
- [x] Documentation has been updated

@czhang771 czhang771 marked this pull request as ready for review October 19, 2024 10:16
@czhang771 czhang771 requested a review from a team as a code owner October 19, 2024 10:16
Copy link
Contributor

@xrmx xrmx left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The sdk part LGTM but I think we are missing a test to assert that sometimes these fields are not serialized

@xrmx
Copy link
Contributor

xrmx commented Nov 6, 2024

The sdk part LGTM but I think we are missing a test to assert that sometimes these fields are not serialized

We need two tests because AFAICS we don't have any tests serializing exemplars in oltp

@fcollonval
Copy link
Contributor

I rebased this PR on main following #4251. @xrmx does your comments still apply to the remaining changes?

@xrmx
Copy link
Contributor

xrmx commented Nov 11, 2024

I rebased this PR on main following #4251. @xrmx does your comments still apply to the remaining changes?

Nope, they were for the sdk changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

Metrics: Add support for exemplars
4 participants