[APM] Adds information on excluding data tiers from searches (#4327) (#…

…4415)

* initial draft

* address feedback

* add feedback

---------

Co-authored-by: Brandon Morelli <[email protected]>
Co-authored-by: Brandon Morelli <[email protected]>
(cherry picked from commit 8ba52b9)

Co-authored-by: Colleen McGinnis <[email protected]>

docs/en/observability/apm/view-and-analyze/filter-and-search/cross-cluster-search.asciidoc

@@ -21,7 +21,7 @@ and allowing for better performance while managing multiple observability use ca
 If you're using the Hosted {ess}, see {cloud}/ec-enable-ccs.html[Enable cross-cluster search].
 
 // lint ignore elasticsearch
-You can add remote clusters directly in {kib}, under *Management* > *Elasticsearch* > *Remote clusters*.
+You can add remote clusters directly in {kib}, under *Management* → *Elasticsearch* → *Remote clusters*.
 All you need is a name for the remote cluster and the seed node(s).
 Remember the names of your remote clusters, you'll need them in step two.
 See {ref}/ccr-getting-started.html[managing remote clusters] for detailed information on the setup process.
@@ -48,3 +48,16 @@ There are two ways to edit the default {data-source}:
 include remote clusters.
 * In `kibana.yml` -- Update the {kibana-ref}/apm-settings-kb.html[`xpack.apm.indices.*`] configuration values to
 include remote clusters.
+
+[TIP]
+.Exclude data tiers from search
+====
+In a cross-cluster search (CCS) environment, it's possible for different clusters to serve different data tiers in responses.
+If one of the requested clusters responds slowly, it can cause a timeout at the proxy after 320 seconds. 
+This results in 502 Bad Gateway server error responses presented as failure toast messages in the UI, and no data loaded.
+
+To prevent this, you can exclude {ref}/data-tiers.html[data tiers] that might slow down responses from search: the `data_frozen` and `data_cold` tiers. To exclude data tiers from search in the APM UI:
+
+. In {kib}, go to *Stack Management* → *Advanced Settings*.
+. In the *Observability* section, update the *Excluded data tiers from search* option with a list of data tiers.
+====