Revise configuring-metricbeat documentation

[matschaffer]

As called out on this forum post the configuring-metricbeat doc for elasticsearch is quite confusing.

1. You shouldn't need to change any ES settings (except maybe to remove legacy settings)
2. There's some disagreement between this doc and https://github.com/elastic/beats/blob/main/metricbeat/module/elasticsearch/_meta/docs.asciidoc regarding which nodes the LB should point to when running in scope: cluster

Also, internally we use scope: node, so I'm wondering if we should make that the recommended configuration.

I'm just guessing, but I think the original scope: cluster + non-master recommendation was made with the idea that we would try not to burden the master with monitoring info.

But the non-master would have to retrieve cluster information from the master internally to satisfy the request either way.

[elasticmachine]

Pinging @elastic/infra-monitoring-ui (Team:Infra Monitoring UI)

[matschaffer]

See https://discuss.elastic.co/t/documentation-for-monitoring-elasticsearch-with-metricbeat-is-outdated/315804 for additional doc feedback.

[DaveCTurner]

But the non-master would have to retrieve cluster information from the master internally to satisfy the request either way.

This is not correct. Most stats requests are peer-to-peer things that do not need to involve the master node, and for the ones which do involve the master node in some way we typically find ways to streamline that involvement in newer versions (see e.g. elastic/elasticsearch#101805). Even offloading the JSON rendering to other nodes can be valuable in large enough clusters.

Also, internally we use scope: node, so I'm wondering if we should make that the recommended configuration.

While I agree that we should internally use the configuration we recommend to others, we can solve that by either changing the recommendation or by changing the configuration we use internally. This isn't the place to discuss that, but ask me privately if you'd like more context :)

TLDR the recommendation to avoid sending stats requests to dedicated master nodes is as valid today as ever.