[REQUEST]: Review the artitecture of Fleet in air-gapped environment

[lucabelluccini]

Description

The page https://www.elastic.co/guide/en/fleet/current/air-gapped.html should be enhanced a little bit more.

Resources

The versions specified below might not be 100% correct.

Kibana needs to:

• Reach EPR
  • https://epr.elastic.co allowing the network access
  • Using a proxy via Kibana settings
  • Self-hosting EPR and via Kibana settings to point to it
• 8.10+ Reach https://www.elastic.co/api/product_versions to get the latest versions available (falls back on a static option)
  • allowing the network access to https://www.elastic.co/api/product_versions
  • (there is no way around this except maybe using HTTP_PROXY/HTTPS_PROXY/NO_PROXY on Kibana env vars to grant access to that domain)

Elastic Agents, need to:

• Reach the Artifact Repository (aka Source URI) to download the binaries (typically upgrading)
  • allowing the network access https://artifacts.elastic.co
  • 8.9+ using an HTTP Proxy set at policy level and still pointing to https://artifacts.elastic.co
  • A self hosted Artifact repository set at policy level
  • 8.9+ A self hosted Artifact repository using an HTTP Proxy set at policy level
• Reach the Fleet Server (to report its status and receive actions to do and policies)
  • allowing the network access to the Fleet Server endpoint
  • defining an HTTP Proxy at policy level to access Fleet Server
  • via cli --proxy-url & co...
• Reach the Output (to send the integration data and the monitoring data)
  • allowing the access to the output endpoint
  • 8.9+ Defining an HTTP Proxy at policy level (⚠ only specific output types, plus some integrations might not support it)
• Reach the PGP/GPG endpoint
  • allowing the network access https://artifacts.elastic.co/GPG-KEY-elastic-agent
  • Reach Fleet Server, setting it up via [REQUEST]: Document PGP key download from Fleet Server API in air-gapped envs #980
• Reach 3rd party endpoints (e.g. AWS, ...) requested by integrations
  • Integrations might allow to provide a proxy URL in the integration configuration
  • If not, it is necessary to rely on the HTTP_PROXY / HTTPS_PROXY / NO_PROXY to allow access (it only works for HTTP protocols).

Fleet Servers need the same accesses of Elastic Agents, plus:

• (seems only on K8s), reach Kibana to get the default policy (?) ⚠ to be clarified.

We might also mention the it's possible to use the HTTP_PROXY / HTTPS_PROXY / NO_PROXY settings (docs) instead of the strategies above. The env var afaik "prevails" on the Proxy set in the policy (I've not verified recently but Craig suggested so some time ago).

By "air-gapped" we mean: no internet access or with network restrictions to several/all external resources mentioned here.

I would recommend to change the page structure in sections per-product and going over the destinations.

Air-gapped environments

• Upgrading in air-gapped environments
• Preparing Kibana to be air-gapped
  • Air gapped mode
  • Configure the access to EPR
    • Via Proxy (xpack.fleet.registryProxyUrl)
    • Via self-hosting EPR (xpack.fleet.registryUrl)
      • Self-hosting EPR
      • Use NODE_EXTRA_CA_CERTS if EPR is exposed via TLS with custom CA
• Preparing Elastic Agents to be air-gapped
  • All external HTTP connections
    • via env vars HTTP_PROXY and HTTPS_PROXY
  • Access to Fleet Server
    • allowing the network access to the Fleet Server endpoint
    • defining an HTTP Proxy at policy level to access Fleet Server
    • via CLI --proxy-url & co
  • Access to Artifact Repository / Source URI
    • allowing the network access to the public one
    • defining an HTTP Proxy and the Source URI (the public one or the self-hosted one) at policy level
      • Self-hosting the Artifact Repository
  • Access to the output
  • allowing the network access to it
  • defining an HTTP Proxy for the output
  • Reach the PGP/GPG endpoint
    • allowing the network access to it
    • allowing access to Fleet Server (preparing Fleet Server to serve the key)

Collaboration

TBD. The docs and product team will work together to determine the best path forward.

Point of contact.

Main contact: @lucabelluccini

Stakeholders:

[kilfoyle]

Just so it's linked here, Nima started a gdoc where we can iterate on this.