Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Collection Kepler metrics using the Prometheus Receiver #30

Open
AdrianHammond opened this issue Nov 22, 2023 · 3 comments
Open

Collection Kepler metrics using the Prometheus Receiver #30

AdrianHammond opened this issue Nov 22, 2023 · 3 comments

Comments

@AdrianHammond
Copy link

I want to use the prometheus collector to receive Keler. The Kepler Operator installs an exporter on each node.

To get around issue with unknown prometheus receiver I have updated the collector POD image to use this image

Below is the YAML I am using for the collector

spec:
  config: |
    receivers:
      prometheus:
        target_allocator:
          endpoint: http://kepler-exporter-svc.openshift-kepler-operator.svc.cluster.local:9103/metrics
          interval: 30s
          collector_id: collector-1

    exporters:
      logging:

    service:
      pipelines:
        metrics:
          receivers: [prometheus]
          exporters: [logging]

I am now seeing this error.

2023-11-22T14:35:59.951Z info [email protected]/telemetry.go:85 Setting up own telemetry...
2023-11-22T14:35:59.951Z info [email protected]/telemetry.go:202 Serving Prometheus metrics {"address": ":8888", "level": "Basic"}
2023-11-22T14:35:59.951Z info [email protected]/exporter.go:275 Deprecated component. Will be removed in future releases. {"kind": "exporter", "data_type": "metrics", "name": "logging"}
2023-11-22T14:35:59.953Z info [email protected]/service.go:143 Starting otelcol-contrib... {"Version": "0.89.0", "NumCPU": 12}
2023-11-22T14:35:59.953Z info extensions/extensions.go:34 Starting extensions...
2023-11-22T14:35:59.953Z info [email protected]/metrics_receiver.go:105 Starting target allocator discovery {"kind": "receiver", "name": "prometheus", "data_type": "metrics"}
2023-11-22T14:35:59.953Z info [email protected]/metrics_receiver.go:239 Starting discovery manager {"kind": "receiver", "name": "prometheus", "data_type": "metrics"}
2023-11-22T14:35:59.961Z error [email protected]/metrics_receiver.go:138 Failed to retrieve job list {"kind": "receiver", "name": "prometheus", "data_type": "metrics", "error": "yaml: unmarshal errors:\n line 1: cannot unmarshal !!str `<html> ...` into map[string]*config.ScrapeConfig"}
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver.(*pReceiver).syncTargetAllocator
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/metrics_receiver.go:138
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver.(*pReceiver).startTargetAllocator
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/metrics_receiver.go:107
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/prometheusreceiver.(*pReceiver).Start
github.com/open-telemetry/opentelemetry-collector-contrib/receiver/[email protected]/metrics_receiver.go:91
go.opentelemetry.io/collector/service/internal/graph.(*Graph).StartAll
go.opentelemetry.io/collector/[email protected]/internal/graph/graph.go:391
go.opentelemetry.io/collector/service.(*Service).Start
go.opentelemetry.io/collector/[email protected]/service.go:161
go.opentelemetry.io/collector/otelcol.(*Collector).setupConfigurationComponents
go.opentelemetry.io/collector/[email protected]/collector.go:190
go.opentelemetry.io/collector/otelcol.(*Collector).Run
go.opentelemetry.io/collector/[email protected]/collector.go:229
go.opentelemetry.io/collector/otelcol.NewCommand.func1
go.opentelemetry.io/collector/[email protected]/command.go:27
github.com/spf13/cobra.(*Command).execute
github.com/spf13/[email protected]/command.go:983
github.com/spf13/cobra.(*Command).ExecuteC
github.com/spf13/[email protected]/command.go:1115
github.com/spf13/cobra.(*Command).Execute
github.com/spf13/[email protected]/command.go:1039
main.runInteractive
github.com/open-telemetry/opentelemetry-collector-releases/contrib/main.go:27
main.run
github.com/open-telemetry/opentelemetry-collector-releases/contrib/main_others.go:11
main.main
github.com/open-telemetry/opentelemetry-collector-releases/contrib/main.go:20
runtime.main
runtime/proc.go:267
2023-11-22T14:35:59.961Z info [email protected]/service.go:178 Starting shutdown...
2023-11-22T14:35:59.961Z info extensions/extensions.go:52 Stopping extensions...
2023-11-22T14:35:59.961Z info [email protected]/service.go:192 Shutdown complete.
Error: cannot start pipelines: yaml: unmarshal errors:
line 1: cannot unmarshal !!str `<html> ...` into map[string]*config.ScrapeConfig
2023/11/22 14:35:59 collector server run finished with error: cannot start pipelines: yaml: unmarshal errors:
line 1: cannot unmarshal !!str `<html> ...` into map[string]*config.ScrapeConfig

Please could advise how I resolve this. Thanks
@pavolloffay
Copy link
Member

cc) @frzifus

@pavolloffay
Copy link
Member

Maybe this tutorial will help you to fix the setup? https://github.com/pavolloffay/kubecon-na-2023-opentelemetry-kubernetes-metrics-tutorial/blob/main/05-collecting-prometheus-metrics.md

It seems like the configuration is incorrect

@frzifus
Copy link
Collaborator

frzifus commented Nov 22, 2023

Hi @AdrianHammond, can you please share the full CRD and provide us some infos about the Operator and environment you are using? :)

Since the output includes v0.89.0 I assume its the community operator?

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
3 participants
@pavolloffay @frzifus @AdrianHammond