[SecuritySolution] Embedded onboarding flow

[angorayc]

Figma

Users should be able to add integration without leaving the onboarding page.

If the Url the integration card user clicks on starts with /app/integrations, it should be opened in the modal in the onboarding hub.

1. It configures the integration.
2. It recommends setting up a Fleet server if it's not set up.
3. Once Fleet server is set up, it continues to set up the Elastic agent.
4. Once the agent is set up, it checks if data has been received.

Tasks

Preview Give feedback

1. Render EPMApp in the onboarding hub modal.
2. Navigate to the CreateIntegration component (included in the EPMApp) when creating the integration.
3. Add agent step should be available when users selecting New hosts in the Configure integration step, and disappear if existing hosts is selected

[elasticmachine]

Pinging @elastic/security-threat-hunting-explore (Team:Threat Hunting:Explore)

[angorayc]

The instructions of how to do the ingestion on a VM and access the agent enrolment page:

https://github.com/elastic/kibana/blob/main/x-pack/plugins/fleet/dev_docs/local_setup/enrolling_agents.md

[angorayc]

How it's different from the existing flow: https://docs.google.com/document/d/1ymeDKagKWpQSUUkd8-F9ij8X-SLtTsre878KC05XRbM/edit?tab=t.0

[angorayc]

Integration flow: https://docs.google.com/document/d/1gmlGCkx5lpjPowcCpn6cCnl39qhRwjj82iZHV5Mj7T4/edit?tab=t.9bw8nshlw37s

How to test the flow : self-managed (Require to set up Fleet server manually) + enroll agent with Fleet

yarn es snapshot --license trial -E xpack.security.authc.api_key.enabled=true -E discovery.type=single-node -E network.host=0.0.0.0
yarn start --no-base-path

1. Enable Docker desktop
2. Find your {xpack.fleet.outputs.hosts} by commend

ipconfig getifaddr en0

the output is your {xpack.fleet.outputs.hosts} in kibana.dev.yaml

In kibana.dev.yml:

# Use default version resolution to let APIs work without version header
server.versioned.versionResolution: oldest

# Install Fleet Server package.
xpack.fleet.packages:
  - name: fleet_server
    version: latest

# Create an agent policy for Fleet Server.
xpack.fleet.agentPolicies:
  - name: Fleet Server policy
    id: fleet-server-policy
    is_default_fleet_server: true
    # is_managed: true # Useful to mimic cloud environment
    description: Fleet server policy
    namespace: default
    package_policies:
      - name: Fleet Server
        package:
          name: fleet_server
        inputs:
          - type: fleet-server
            keep_enabled: true
            vars:
              - name: host
                value: 0.0.0.0
                frozen: true
              - name: port
                value: 8220
                frozen: true

# Set a default Fleet Server host.
xpack.fleet.fleetServerHosts:
  - id: default-fleet-server
    name: Default Fleet server
    is_default: true
    host_urls: [https://{xpack.fleet.fleetServerHosts.host_urls}:8220] # For running a Fleet Server in a VM
    # host_urls: ['https://host.docker.internal:8220'] # For running a Fleet Server Docker container

# Set a default Elasticsearch output.
xpack.fleet.outputs:
  - id: es-default-output
    name: Default output
    type: elasticsearch
    is_default: true
    is_default_monitoring: true
    hosts: ['http://{xpack.fleet.outputs.hosts}:9200'] # For enrolling agents on VM
    # hosts: ['http://host.docker.internal:9200'] # For enrolling dockerized agents

Then start the fleet server:

cd ./kibana 
node x-pack/plugins/security_solution/scripts/endpoint/start_fleet_server.js

Once fleet server is started:
Visit /app/fleet/settings
Find the newly added Fleet server url in the list

Then in kibana.dev.yml
Update your xpack.fleet.fleetServerHosts.host_urls in kibana.dev.yml

Visit /app/fleet/policies
Create a new policy, in Advanced options, select the Fleet Server you just found

Back to /app/fleet/policies table , find the policy you just created, on the rightest column, and select Add agent

In the add agent flout, you would be able to find FLEET_SERVER_URL and TOKEN, and put them in the command below

Open a new terminal:

try below commends to enroll your agent:

docker run  --add-host host.docker.internal:host-gateway  \
  --env FLEET_ENROLL=1 --env FLEET_INSECURE=true \
  --env FLEET_URL=https://host.docker.internal:8220 \
  --env FLEET_ENROLLMENT_TOKEN={TOKEN} \
  --user=root \
  docker.elastic.co/beats/elastic-agent:8.15.3

if not working, try this:

docker run  --add-host host.docker.internal:host-gateway  \
  --env FLEET_ENROLL=1 --env FLEET_INSECURE=true \
  --env FLEET_URL=https://{FLEET_SERVER_URL}:8220 \
  --env FLEET_ENROLLMENT_TOKEN={TOKEN}  \
  --user=root \
  docker.elastic.co/beats/elastic-agent:8.15.3

How to test the flow : running a standalone agent

Prerequisite:

Download UTM: https://mac.getutm.app/ > Download
Download Ubuntu image: (image: https://mac.getutm.app/gallery/ubuntu-20-04)

0. Enable clipboard sharing: https://docs.getutm.app/guest-support/sharing/clipboard/
1. Edit the network settings for the VM. On the side Navigation of the UTM, select the image, right click > Edit > Network > Network Mode: Emulated VLAN
2. Start the VM

curl -L -O https://artifacts.elastic.co/downloads/beats/elastic-agent/elastic-agent-8.17.0-linux-arm64.tar.gz ### NOT x86_64
tar xzvf elastic-agent-8.17.0-linux-arm64.tar.gz
cd elastic-agent-8.17.0-linux-arm64

update elastic-agent.yml according to the setup in Fleet

sudo ./elastic-agent install

[angorayc]

• Security Solution onboarding hub UI changes - @kapral18 [Security Solution] Clean-up embedded onboarding flow UI #201402
• Fleet UI changes @angorayc

[angorayc]

Pending questions:

Missing screens :

1. A screen to ask if you want to enroll the agent with Fleet? Yes -> set up fleet if required. No -> run standalone agent.
2. A screen to set up Fleet server
3. A screen to set up standalone agent only

Not sure about how to deal with the actions in the integration tabs: