Prometheus target for Routers

[sayeedc]

I setup a separate prometheus instance to scrape the targets for haproxy, but wasn't able to pull any data. Can provide the prometheus config that's required for this setup?

[luszczynski]

Hey @sayeedc,

If you take a look at the Prometheus CR [1] you will see all the details that I used to setup my prometheus instance.
Are you facing any kind of error?

[1] https://github.com/luszczynski/openshift-router-monitoring/blob/master/prometheus-operator-template.yaml#L205

[sayeedc]

Hey @luszczynski

I've got a 401 error when trying to scrape the router metrics from prometheus. I'm using an existing one that's deployed via helm charts, and only need to pull in the HA proxy metrics. To view the kube-state metrics, I actually use the prometheus server in the OCP3.11 cluster monitoring stack as a datasource in my Grafana instance - I can't make modifications to the OCP 3.11 cluster monitoring stack.

I've setup a target on my prometheus server

      - job_name: 'router-apps'
        static_configs:
          - targets:
              - router-apps.default.svc.cluster.local:1936
        basic_auth:
          username: <omitted>
          password: <omitted>

The service monitor is running in the same namespace as my prometheus instance:

apiVersion: monitoring.coreos.com/v1
kind: ServiceMonitor
metadata:
  labels:
    metrics: router-metrics
  name: router-metrics
  namespace: <my-prometheus namespace>
spec:
  endpoints:
  - basicAuth:
      password:
        key: password
        name: servicemonitor-auth
      username:
        key: username
        name: servicemonitor-auth
    interval: 5s
    path: /metrics
    port: monitoring
    scheme: http
  namespaceSelector:
    matchNames:
    - default
  selector:
    matchLabels:
      app: router-apps

I can curl the metrics on the router-apps using the same credentials from within the prometheus pod, but there's something missing on the target settings that's not allowing prometheus to scrape the same endpoint. What am I missing?