Clarifying metric yam configuration file for ksql

[okouyad]

I would like to understand the meaning of the comments on top of the metrics extraction as in this example

  #"kafka.consumer:type=consumer-node-metrics,client-id=*, node-id=*"
  # "kafka.consumer:type=consumer-fetch-manager-metrics,client-id=*, topic=*"
  # "kafka.producer:type=producer-node-metrics,client-id=*, node-id=*"
  # "kafka.producer:type=producer-topic-metrics,client-id=*, topic=*"
  - pattern: "kafka.(.+)<type=(.+), (.+)=(.+), (.+)=(.+)><>(.+):"
    name: kafka_$1_$2_$7
    type: GAUGE
    labels:
      client_type: $1
      $3: "$4"
      $5: "$6"

Do they stand for example of metrics extracted by the rule ?

Where i get confused is that, if that is the case why is it that at the top of the file we have in the blacklist the following:
blacklistObjectNames:

  - "io.confluent.ksql.metrics:name=*"
  - kafka.streams:type=kafka-metrics-count
  # This will ignore the admin client metrics from KSQL server and will blacklist certain metrics
  # that do not make sense for ingestion.
  - "kafka.admin.client:*"
  - "kafka.consumer:type=*,id=*"
  - "kafka.consumer:type=*,client-id=*"
  - "kafka.consumer:type=*,client-id=*,node-id=*"
  - "kafka.producer:type=*,id=*"
  - "kafka.producer:type=*,client-id=*"
  - "kafka.producer:type=*,client-id=*,node-id=*"
  - "kafka.streams:type=stream-processor-node-metrics,thread-id=*,task-id=*,processor-node-id=*"
  - "kafka.*:type=kafka-metrics-count,*"
  - "io.confluent.ksql.metrics:type=_confluent-ksql-rest-app-command-runner,*"

It feels like a contradiction to me.

I would like to inspire myself from the file, but i find it confusing at time. It seems we are blacklisting everything about consumer and producer, yet the rule below show how some of them might extract the very pattern that is blacklisted.

Can someone help me clear my confusion please

https://github.com/confluentinc/jmx-monitoring-stacks/blob/7.1-post/shared-assets/jmx-exporter/confluent_ksql.yml

[dberardo-com]

i have noticed that the exported metrics do not match up with the dashboard quantities in grafana ... is there a way to map it ??

[hifly81]

@dberardo-com could you provide more details pls?