Can't find a clear example of sending metrics or traces to prometheus

[montaropdf]

Hello,

I am, currently, studying and evaluating opentelemetry for integration to an application and how it would fit into the infrastructure.

I have searched, for some time, how to use the opentelemetry-exporter-prometheus to send metrics and traces to a prometheus instance, but can't find any example that actually show how to generate and send the data to prometheus.

The only example I have found is https://opentelemetry.io/docs/instrumentation/python/exporters/#prometheus.

When I add code to actually generate metrics, the prometheus instance receives nothing, it seems.

Below is the code I use to generate metrics using opentelepetry and the prometheus exporter.

from prometheus_client import start_http_server

from opentelemetry import metrics
from opentelemetry.exporter.prometheus import PrometheusMetricReader
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader
from opentelemetry.sdk.resources import SERVICE_NAME, Resource


if __name__ == '__main__':
    # Service name is required for most backends
    resource = Resource(attributes={
        SERVICE_NAME: "otel-prom-test"
    })

    # Start Prometheus client
    start_http_server(port=8000, addr="localhost")
    # Initialize PrometheusMetricReader which pulls metrics from the SDK
    # on-demand to respond to scrape requests
    reader = PrometheusMetricReader()
    provider = MeterProvider(resource=resource, metric_readers=[reader])
    metrics.set_meter_provider(provider)

    meter = metrics.get_meter_provider().get_meter("getting-started", "0.1.2")

    # Counter
    print("Synchronous counter")
    counter = meter.create_counter("counter")
    counter.add(1)
    print("Sleeping for 3 secs.")
    time.sleep(3)
    counter.add(1)
    print("Finished counting.")

What am I missing?

[ocelotl]

I followed this example to start Prometheus in a docker container. I used demo0 as the value for job_name in the prometheus.yml file.

Once it was running I executed your code (I added the missing import time). Prometheus received data:

[jaggadhruv]

@jaggadhruv does the simple example with one metric work for you? When the state is down is your application running? Is "http://localhost:8000/metrics" available at that time?

When the state is down the application is still running and localhost:8000 is also available.

[jaggadhruv]

Ah, yes. You need to listen on all interfaces

    start_http_server(port=8000, addr="0.0.0.0")

I will check this

[jaggadhruv]

Ah, yes. You need to listen on all interfaces

    start_http_server(port=8000, addr="0.0.0.0")

I will check this

This helps a bit, so now I can scrape some data, but still the counter values I can not see, even with 1 counter active

[thiromi]

I had the same problem of my metrics not showing up and it turned out I was running the application with the agent (through opentelemetry-instrument python yourapp.py). You should see the metrics in the /metrics endpoint if you run the script as usual: python yourapp.py.

That's because there's no way in the current version to set a new meter provider whenever using the agent or change the readers already set in the MeterProvider, so you'll need to set up opentelemetry manually if you want to export to Prometheus. Also, there's no way to set Prometheus as a metrics exporter through the agent config (well, I tried to install the opentelemetry-exporter-prometheus package and run the agent with OTEL_METRICS_EXPORTER="prometheus" but I had no luck).

[mmanikan]

Do you find any resolution for this ?