[Stack Monitoring] remove cluster_stats dependency

[klacabane]

Summary

Related #125401

Dirty implementation of a Stack Monitoring app that doesn't rely on elasticsearch.cluster_stats to display other monitored stack products.

• the /clusters API call is now executing a single query that returns information of each monitored products, grouped by cluster_uuid. This data is shown is the Cluster listing view. I've removed the Indices count and Data size from the results since I don't see this data as valuable at this stage of the journey, and I'd rather have an overview of which products are actually tied to this cluster.
  The query relies on a single metricset (per product) being ingested to flag a product as monitored. This may sound fragile because the elasticsearch module collects 9 metricsets to power Stack Monitoring but this is reasonable because the metricsets are hardcoded when xpack.enabled: true, and the absence of the other metricsets would be an operational issue. We can implement these improvements [Stack Monitoring] Tolerate lack of elasticsearch metricset #130577 to cover this scenario gracefully.

• remove the hard dependency on cluster_stats documents in the /clusters/<uuid> API. Note that this endpoint can be refactored more deeply and doesn't need to handle multiple clusters anymore since it is handled by the previous point.
• mock the supported license. This one is a product of laziness, I've only mocked the fact that each returned clusters have a supported license but if we move forward with this, any code around isSupported checks or license verification will be obsolete and can be safely removed.

Testing

• Start a standalone metricbeat[1]
• Start a metricbeat that monitors Kibana and the standalone Metricbeat previously started[2]
• Navigate to Stack Monitoring. You should see two entries in the Cluster listing
• Navigate to the cluster that reports one Kibana instance and verify the UI loads without Elasticsearch data

[1] standalone metricbeat

path.home: ./beats-monitor
http.port: 5066
http.enabled: true

metricbeat.modules:
  - module: system

output.console:
  pretty: true

[2] monitoring metricbeat

http.enabled: true
http.port: 5067

metricbeat.modules:
  - module: kibana
    xpack.enabled: true
    period: 10s
    hosts: [ "http://localhost:5601" ]
    basepath: "/<basepath>"
    ssl.verification_mode: none
    username: "elastic"
    password: "changeme"
  - module: beat
    xpack.enabled: true
    period: 10s
    hosts: ["http://localhost:5066"]

output.elasticsearch:
  hosts: [ "http://localhost:9200" ]
  username: "elastic"
  password: "changeme"
  allow_older_versions: true
  ssl.verification_mode: none

[kibana-ci]

💔 Build Failed

• Buildkite Build
• Commit: d1b8279
• Interpreting CI Failures

Failed CI Steps

• FTR Configs #14
• FTR Configs #14
• FTR Configs #57
• FTR Configs #57
• Serverless Security Tests
• Linting
• Checks

Test Failures

• [job] [logs] FTR Configs #14 / Monitoring app Cluster listing with trial license clusters "before all" hook in "with trial license clusters"
• [job] [logs] FTR Configs #14 / Monitoring app Cluster listing with trial license clusters "before all" hook in "with trial license clusters"
• [job] [logs] FTR Configs #57 / Monitoring cluster list with trial license clusters should load multiple clusters
• [job] [logs] FTR Configs #57 / Monitoring cluster list with trial license clusters should load multiple clusters

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
monitoring	462.4KB	462.5KB	+198.0B

History

• 💔 Build #93304 failed 1d31605
• 💔 Build #93289 failed e02f7f4

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream