[Security Solution] Allow users to edit related_integrations field for custom rules

[maximpn]

Resolves: #173595

Summary

This PR adds an ability to add and edit custom rule's related integrations. Functionality is necessary to start working on Prebuilt Rule Customization Epic Milestone 3.

Details

Rule's related integrations represent optional dependencies on Elastic integrations to ingest data. Currently prebuilt rule's related integrations are shown on rule details page. This information contains integration's name, installation status and a version mismatch warning when related integration's version dependency doesn't match with an installed integration's version. A subset of Semver is used to specify version dependency. Elastic prebuilt rules use only caret syntax like ^1.2.3.

To make it possible to add and edit related integrations for custom rules the following has been done

• New internal endpoint /internal/detection_engine/fleet/integrations/all has been added. It returns the full list of available integrations containing title, latest available version and installed version if available. This is necessary to display an options list where users can pick a desired integration. Since some Elastic Prebuilt rules depend not only on integrations from security category this endpoint returns all available integrations (not only related to Security Solution).
• Rule create form has been adjusted by adding Related Integrations form controls
• Rule edit form has been adjusted by adding Related Integrations form controls
• Related integrations installation status has been adjusted to conform with the design
• Functional Jest tests have been added
• Functional tests have been added to make sure it's possible to (bulk) create/patch/update/export/import with related integrations
• A limited number of Cypress tests have been added

Integration installation status

Integration installation status has been adjusted. There are following statuses shown

• Enabled for installed and enabled integrations. Enabled integrations are detected by checking Elastic Agent policies for presence of such an integration. It's not guaranteed the policy is picked by agents and data is being ingested.
• Disabled for installed and disabled integrations. An agent policy containing such an integration isn't found.
• Not installed for not installed integrations.
• Nothing is shown for unknown integrations. If there is no such integration found in /internal/detection_engine/fleet/integrations/all result it's considered as unknown.

Version dependency

Semver allows a wide range of version range declaration. Such flexibility will complicate constructing of an integration link on rule details page. Since Elastic Prebuilt rules use only caret version dependency like ^1.2.3 related integration's version dependency is limited to a subset of semver semantic. The following is supported

• A plain version dependency e.g. 1.2.3
• Tilde version dependency e.g. ~1.2.3
• Caret version dependency e.g. ^1.2.3

Misc

• [Security Solution] Related Integrations title doesn't always match title shown in Fleet Integrations UI #152408 has been fixed by this PR.
• /internal/detection_engine/fleet/integrations/installed endpoint hasn't been removed. We need to make sure it's not needed anymore.
• E2e testing of the current functionality is complicated by dependency on EPR and difficulties to mock it. EPR periodically may respond with an error resulting in flaky Cypress tests.

Flaky test runner results

• 🟢 Create rule (100 runs ESS and 100 runs in Serverless)
• 🟢 Rule Management related integrations (100 runs ESS and 100 runs in Serverless)

Screenshots

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[kibana-ci]

💚 Build Succeeded

• Buildkite Build
• Commit: 82acefe
• Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-178295-82acefe1d93b
• Security Deployment

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
securitySolution	5466	5477	+11

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
lists	141.1KB	141.2KB	+72.0B
securitySolution	13.7MB	13.7MB	+9.6KB
total			+9.7KB

Canvas Sharable Runtime

The Canvas "shareable runtime" is an bundle produced to enable running Canvas workpads outside of Kibana. This bundle is included in third-party webpages that embed canvas and therefor should be as slim as possible.

id	before	after	diff
module count	-	5407	+5407
total size	-	9.1MB	+9.1MB

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
core	407.0KB	407.1KB	+72.0B

History

• 💚 Build #207415 succeeded 7f08e435829357c98f02c0f959837afe469b5912
• 💛 Build #207383 was flaky 395aceab6cc29050d2018b005da153a7e19650e8
• 💔 Build #207309 failed 71cc73280b62fd293c1137bcbc3b9eb58ef6bb7f
• 💔 Build #207145 failed ca049ffd16757104bc5009e202d26a01b350d6f5
• 💔 Build #207060 failed d41c45f00edfd6c2878222b551ee66b350c035b2

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @maximpn