Upgrade prometheus

[thampiotr]

PR Description

This upgrades the Prometheus dependency. Some required, notable changes:

• scrape_protocols was added to prometheus.scrape
• enable_protobuf_negotiation will remain working the same as previous, but it is now deprecated and scrape_protocols should be used instead.
• Loki had to be updated since the version we used depended on incompatible Prometheus.
• Had to fork github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver as it depended on an incompatible Prometheus version.
• Had to fork go.opentelemetry.io/collector/featuregate as we had to work around the problem where Prometheus and OTEL try to register duplicate OTEL feature flags and result in panics.
• Had to change the way discoverers are created to make sure they correctly register/unregister SD metrics and refresh metrics and pass them over to Prometheus library.
• Some package renames required changes, for example "github.com/prometheus/prometheus/model/textparse" -> "github.com/prometheus/common/model"
• Some of the import changes were needed to add "/v3/" to Prometheus import paths. There are some more changes and shuffles due to merge conflicts.
• Added support for AppendCTZeroSample method in appenders where needed. It is not exposed in any config right now to keep this PR smaller in scope.

Which issue(s) this PR fixes

Fixes #227

Notes to reviewer

Tested this with adding / removing discoverers at runtime and checking that their metrics are unregistered and no panics happen. For example one test scenario:

discovery.docker "containers" {
  host = "unix:///var/run/docker.sock"
}

discovery.file "local" {
  files = ["./file_discovery.yml"]
}

prometheus.scrape "default" {
  targets    = concat(discovery.docker.containers.targets, discovery.file.local.targets)
  forward_to = [prometheus.remote_write.cloud.receiver]
}

We can see these metrics:

➜  discoverers git:(main) ✗ curl -s localhost:12345/metrics | grep disco
prometheus_sd_file_read_errors_total{component_id="discovery.file.local",component_path="/"} 0
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local",component_path="/",quantile="0.5"} 5.9375e-05
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local",component_path="/",quantile="0.9"} 0.000206667
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local",component_path="/",quantile="0.99"} 0.000206667
prometheus_sd_file_scan_duration_seconds_sum{component_id="discovery.file.local",component_path="/"} 0.000266042
prometheus_sd_file_scan_duration_seconds_count{component_id="discovery.file.local",component_path="/"} 2
prometheus_sd_file_watcher_errors_total{component_id="discovery.file.local",component_path="/"} 0
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.5"} 0.085011917
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.9"} 0.244720042
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.99"} 0.244720042
prometheus_sd_refresh_duration_seconds_sum{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 0.7002042099999999
prometheus_sd_refresh_duration_seconds_count{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 7
prometheus_sd_refresh_failures_total{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 0

After renaming the file discovery component and reloading the config we can see:

➜  discoverers git:(main) ✗ curl -s localhost:12345/metrics | grep disco
prometheus_sd_file_read_errors_total{component_id="discovery.file.local_renamed",component_path="/"} 0
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local_renamed",component_path="/",quantile="0.5"} 0.00023325
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local_renamed",component_path="/",quantile="0.9"} 0.00023325
prometheus_sd_file_scan_duration_seconds{component_id="discovery.file.local_renamed",component_path="/",quantile="0.99"} 0.00023325
prometheus_sd_file_scan_duration_seconds_sum{component_id="discovery.file.local_renamed",component_path="/"} 0.00023325
prometheus_sd_file_scan_duration_seconds_count{component_id="discovery.file.local_renamed",component_path="/"} 1
prometheus_sd_file_watcher_errors_total{component_id="discovery.file.local_renamed",component_path="/"} 0
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.5"} 0.085011917
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.9"} 0.244720042
prometheus_sd_refresh_duration_seconds{component_id="discovery.docker.containers",component_path="/",mechanism="docker",quantile="0.99"} 0.244720042
prometheus_sd_refresh_duration_seconds_sum{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 0.7002042099999999
prometheus_sd_refresh_duration_seconds_count{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 7
prometheus_sd_refresh_failures_total{component_id="discovery.docker.containers",component_path="/",mechanism="docker"} 0

PR Checklist

• CHANGELOG.md updated
• Documentation added
• Tests updated
• Config converters updated

[ptodev]

Thank you! I added a few minor comments. As long as Matt's comments are also addressed, I'd be happy to approve it.

[mattdurham]

Is this from some existing code? Looking at the prometheus code I don't see similar logic.

[thampiotr]

There's a new appender method AppendCTZeroSample and I'm implementing it here in fanout to match the way other appender methods are implemented. This is not copied from anywhere.

[mattdurham]

Looks good, maybe a question on how ctzappender is used but I am not up to date on what it really does.

[thampiotr]

Looks good, maybe a question on how ctzappender is used but I am not up to date on what it really does.

I believe you cannot even use it in Alloy right now, you'd need to enabled it via https://github.com/prometheus/prometheus/blob/07355c9199e0913a76e1a992816e63b0683d16e7/cmd/prometheus/main.go#L224 - which we are not yet exposing. It's for the timestamp of the creation of the series, which may or may not become a popular feature: https://github.com/prometheus/proposals/blob/main/proposals/2023-06-13_created-timestamp.md