[Meta] Observability alerting doc improvements

[dedemorton]

This meta issue tracks all the issues and tasks related to improving the Observability alerting documentation.

The work required to resolve this issue will span multiple sprints.

• [REQUEST] Alerting Documentation Improvements #2605.
• Additional changes for stateful docs:
  • Update stack rules by removing links to observability rules and replacing them with a single link. Users should be able to find this link easily (shouldn't be buried low on a sub-page). Also the section task about categories that no longer exist. Fixed by [DOCS] Replace table of links with single link to Obs alerting docs kibana#177525
  • Restructure the content -- separate defining rules from viewing alerts -- similar to what we've done in serverless (see draft TOC). Be as consistent as possible with serverless to make porting the changes back and forth easier. Make alerting navigation more consistent with serverless docs #3640
  • Add list of valid connectors to the alerting docs #3659 [Alerting] Add list of valid connectors to alerting rules docs #3690
  • Add list of valid action variables to the alerting docs #3658 [Alerting] Add list of action variables to Observability alerting docs #3693
  • [Alerting]: Add types and more examples to the alerting docs about action variables  #3735
  • [Request]: Add list of valid action variables to APM alerting topics #4040 (related to [Alerting] Add list of action variables to Observability alerting docs #3693).
  • [Request]: Improve organization and structure of docs about creating/managing rules #4128
  • Consider removing screen captures that don't add value. Resize the ones that we decide to keep because they are ridiculously large. Removing this because we can fix the size of images easily after we migrate to docsmobile, so there is no need to do this now
• Additional changes for serverless docs:
  • Add action variables and list of valid connectors to serverless docs (after we've decided on the ideal structure for alerting topics).
  • Test and update the docs about viewing alerts. Looks like there are some options in the actions menu that are not documented. Validate other alerting topics as time allows. [Request]: Test and update the docs about viewing alerts #4039
  • Update stale screen captures in serverless
• [Request]: Add documentation for missing alerting rules in serverless and clean up nav #3534 Fixed by: https://github.com/elastic/staging-serverless-observability-docs/pull/222
• Enhance alerting examples #3899

Open questions

• When syncing the stateful content with serverless, do we want to align with the serverless content as closely as possible? YAS!
• Do we like the format of the serverless content and want to follow it? Maybe - def like the collapsible sections, but want to make sure procedures in serverless do not gloss over details. Also want to remove extraneous screen captures in favor of providing content that adds value.
• How should we handle the Uptime-related rules? Looks like some of them do appear under Synthetics. Do we want update the content to show they are relevant for Synthetics? Are they deprecated? Will they be discontinued? Removed? We are leaving them for now. We may want to add a note to the page to indicate that the Uptime UI is disabled until Heartbeat data is available.
• Are there threshold rules that should be marked as deprecated now that the custom threshold rule is GA? Is it worth the effort right now to rewrite those docs to sync them with serverless? Are they deprecated? Will they be discontinued? Removed? TBD. Leave as-is for now.
• Is there any reason why we are not alphabetizing the list of rule types in the UI? It's hard to see the organizing principle at work here, which means the list is harder for users to scan. This is fixed (in serverless, at least)

Draft TOC (stateful)

• Alerting
  • Create and manage rules
    • Custom threshold
    • Inventory threshold
    • Log threshold
    • Metric threshold
    • Monitor status
    • SLO burn rate
    • TLS certificate
    • Uptime duration anomaly
    • Other* (create one category with pointersor move the pages to appear alphabetically in this section)?
  • View and manage alerts

*Missing from the navigation:

Elasticsearch query - covered in Kibana docs here: https://www.elastic.co/guide/en/kibana/8.12/rule-type-es-query.html
Anomaly detection - covered in the ML docs here: https://www.elastic.co/guide/en/machine-learning/master/ml-configuring-alerts.html
APM Anomaly - covered in APM docs here: https://www.elastic.co/guide/en/kibana/8.12/apm-alerts.html
Error count threshold - covered in APM docs here: https://www.elastic.co/guide/en/kibana/master/apm-alerts.html
Failed transaction rate threshold - covered in APM docs here: https://www.elastic.co/guide/en/kibana/master/apm-alerts.html
Latency threshold - covered in APM docs here: https://www.elastic.co/guide/en/kibana/master/apm-alerts.html

Rules that appear under Observability in 8.13:

[dedemorton]

This meta issue has gotten quite large. I've made sure that we have issues to track the remaining work, so I am going to close this issue now so we don't have to kick this issue forward with every sprint.