[Security Solution] Editing rules independently of source data (#180407) #191487

e40pud · 2024-08-27T11:53:30Z

Summary

Addresses #180407
Addresses #178611

With these changes we allow user to create and update a rule even if there are certain query bar validation error exist. Right now, we will make any non-syntax validation errors in EQL and ES|QL rules types to be non-blocking during the rule creation and rule updating workflows.

Screenshot of the EQL rule creation workflow with existing non-blocking validation errors:

Screen.Recording.2024-08-27.at.13.44.14.mov

Screenshot of the EQL rule updating workflow with existing non-blocking validation errors:

Screen.Recording.2024-08-27.at.13.46.19.mov

UPDATE

After discussing confirmation modal with @approksiu, we decided to simplify it and show only title with generic description to avoid too be too literal in the modal. User can see the full error description during rule creation/editing workflows in the query bar where we show each validation error as part of the query bar form item.

Some test cases for local testing

Create EQL rule with missing data source

Steps:

Open rules management page
Click create new rule button
Select EQL rule type
Set non-existing index in index patterns field
Add some valid EQL query (for example any where true)
Continue with other steps
Click create rule button

Expected: You will see the confirmation modal that warns user about potentially failing rule executions. Clicking Confirm button will create a rule.

Create EQL rule with missing data field

Steps:

Open rules management page
Click create new rule button
Select EQL rule type
Set existing indices in index patterns field
Add some valid EQL query referring non-existing data field (for example any where agent.non_existing_field)
Continue with other steps
Click create rule button

Expected: You will see the confirmation modal that warns user about potentially failing rule executions. Clicking Confirm button will create a rule.

Create EQL rule with syntax error in the query

Steps:

Open rules management page
Click create new rule button
Select EQL rule type
Set existing indices in index patterns field
Add some syntactically invalid EQL query (for example hello world)

Expected: The continue button does not allow user to proceed to the About step due to existing syntax error.

Create ES|QL rule with missing data source

Steps:

Open rules management page
Click create new rule button
Select ES|QL rule type
Add some valid ES|QL query with non-existing data source (for example from non-existing-index-* metadata _id, _version, _index | SORT @timestamp)
Continue with other steps
Click create rule button

Expected: You will see the confirmation modal that warns user about potentially failing rule executions. Clicking Confirm button will create a rule.

Create ES|QL rule with missing data field

Steps:

Open rules management page
Click create new rule button
Select ES|QL rule type
Add some valid ES|QL query with non-existing data field (for example from logs-* metadata _id, _version, _index | SORT agent.non_existing_field)
Continue with other steps
Click create rule button

Expected: You will see the confirmation modal that warns user about potentially failing rule executions. Clicking Confirm button will create a rule.

Create ES|QL rule with syntax error in the query

Steps:

Open rules management page
Click create new rule button
Select ES|QL rule type
Add some syntactically invalid ES|QL query (for example hello world)

Expected: The continue button does not allow user to proceed to the About step due to existing syntax error.

Same behaviour applies to the rule updating workflow. For example, you can try to install one of the EQL or ES|QL rules that point to non-existing data source or uses non-existing data field. User can still update (add rule actions) to such installed pre-built rules.

Checklist

Delete any items that are not applicable to this PR.

Any text added follows EUI's writing guidelines, uses sentence case text and includes i18n support
Documentation was added for features that require explanation or tutorials
- Editing rules independently of source data security-docs#5758
Unit or functional tests were updated or added to match the most common scenarios
Flaky Test Runner was used on any tests changed
- Detection Engine - Cypress (100 ESS & 100 Serverless)
- Rule Management - Cypress (100 ESS & 100 Serverless)

…ic#180407)

e40pud · 2024-08-27T11:53:40Z

/ci

e40pud · 2024-08-27T14:53:41Z

/ci

e40pud · 2024-08-27T15:24:59Z

@elasticmachine merge upstream

e40pud · 2024-08-27T15:25:16Z

/ci

…-fix'

e40pud · 2024-08-27T16:24:51Z

/ci

elasticmachine · 2024-08-27T18:07:42Z

Pinging @elastic/security-solution (Team: SecuritySolution)

elasticmachine · 2024-08-27T18:07:43Z

Pinging @elastic/security-detection-engine (Team:Detection Engine)

e40pud · 2024-08-28T10:53:26Z

@elasticmachine merge upstream

vitaliidm

thanks for implementing this @e40pud
Looks good, just few minor comments from me

x-pack/plugins/security_solution/public/detection_engine/rule_creation_ui/pages/translations.ts

...gins/security_solution/public/detection_engine/rule_creation_ui/pages/rule_editing/index.tsx

x-pack/plugins/security_solution/public/detection_engine/rule_creation_ui/pages/form.tsx

vitaliidm · 2024-08-29T09:13:59Z

...ins/security_solution/public/detection_engine/rule_creation_ui/pages/rule_creation/index.tsx

@@ -559,7 +633,8 @@ const CreateRulePageComponent: React.FC = () => {
  );
  const memoDefineStepExtraAction = useMemo(
    () =>
-      defineStepForm.isValid && (
+      defineStepForm.isValid !== undefined &&
+      activeStep !== RuleStep.defineRule && (


why extra condition is needed here?

added comments

vitaliidm · 2024-08-29T09:26:42Z

.../public/detection_engine/rule_creation_ui/components/save_with_errors_confirmation/index.tsx

+ * 2.0.
+ */
+
+import React from 'react';


@e40pud

is this fine user that user sees some error, but can't go to Definition tab or see error itself?

I imagine it will be a case until full support of prebuilt rule editing will be released.
But before that, it can affect Serverless releases

What you see right now is the current behaviour on both ESS and Serverless. Users are unable to modify prebuilt rules. They have to duplicate prebuilt rules right now to be able to adjust rule's properties.

The whole "rule customization" effort will allow user to do that without duplication.

I am aware, prebuilt rule can't be modified.
It's the fact that we show user step is invalid without showing actual error and additional information how that can affect rule's behavior.

Is there a way for user in this situation to see actual error?

As discussed here is the ticket to discuss this behaviour #191832

x-pack/plugins/security_solution/public/detection_engine/rule_creation_ui/pages/form.tsx

...ins/security_solution/public/detection_engine/rule_creation_ui/pages/rule_creation/index.tsx

…creation_ui/pages/translations.ts Co-authored-by: Vitalii Dmyterko <[email protected]>

e40pud · 2024-08-29T10:02:47Z

@elasticmachine merge upstream

e40pud · 2024-08-29T14:34:32Z

@elasticmachine merge upstream

- memoize callbacks - clarify multiple EQL validation errors functionality - comments to clarify step editing button visibility conditions - add unit tests for `useRuleFormsErrors`

kibana-ci · 2024-08-29T15:18:19Z

💛 Build succeeded, but was flaky

Buildkite Build
Commit: 2cfbf72
Cloud Deployment
Kibana Serverless Image: docker.elastic.co/kibana-ci/kibana-serverless:pr-191487-2cfbf7251ef5
Security Deployment

Failed CI Steps

FTR Configs #59

Metrics [docs]

Module Count

Fewer modules leads to a faster build time

id	before	after	diff
`securitySolution`	5725	5727	+2

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	18.0MB	18.0MB	+7.6KB

History

💚 Build #230724 succeeded 07a60cf
💛 Build #230391 was flaky b907087
💚 Build #230220 succeeded 7b2616a
💛 Build #230187 was flaky 7b2616a

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @e40pud

kibanamachine · 2024-08-30T04:57:46Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6830

[❌] Security Solution Rule Management - Cypress: 75/100 tests passed.
[✅] [Serverless] Security Solution Rule Management - Cypress: 100/100 tests passed.

see run history

kibanamachine · 2024-08-30T18:00:13Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6828

[❌] Security Solution Detection Engine - Cypress: 30/100 tests passed.
[❌] [Serverless] Security Solution Detection Engine - Cypress: 42/100 tests passed.

see run history

kibanamachine · 2024-08-30T21:28:58Z

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6831

[❌] Security Solution Detection Engine - Cypress: 49/100 tests passed.
[❌] [Serverless] Security Solution Detection Engine - Cypress: 36/100 tests passed.

see run history

… editing (#191832) (#192683) ## Summary Partially addressed #191832 With these changes: - We revert to the #180407 (comment). Specifically, we return back the validation errors to the modal window. An example of this modal is in the ticket description. - Additionally, on the Rule Editing page and **only for prebuilt rules** we: 1) hide the callout that says "You have an invalid input in this tab: ...", and 2) we don't show the modal if there are any data validation errors. We shouldn't show this modal and this callout until we release the prebuilt rule customization feature. 3) We will only validate the Actions tab. - Fix MKI flaky cypress tests introduced in #191487 ([1](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/b1f442af-db44-8029-a9fb-7e3d988303b3?branch=main), [2](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/995655b6-ae70-86fd-b483-c65846cd8d66?branch=main), [3](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/02318f5c-6ca1-8779-a5a4-60f52a55b344?branch=main)). All three tests are failing due to missing `[data-test-subj="eqlRuleType"]` element. After checking and comparing my tests to other similar tests in the file, the only difference that I've found was extra `login();` call. Thus removing those. Here is the screen recording showing the new behaviour for prebuilt rules. The has missing data source query validation error, though we do not show it and allow user just to save the rule. Only Actions tab is validated on rule save action. https://github.com/user-attachments/assets/ce968f51-1a53-41b2-ad06-1b31dec085a6 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed * [Detection Engine - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6925) (100 ESS & 100 Serverless) * [Rule Management - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6926) (100 ESS & 100 Serverless) * [Prebuilt Rules - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6927) (100 ESS & 100 Serverless)

… editing (elastic#191832) (elastic#192683) ## Summary Partially addressed elastic#191832 With these changes: - We revert to the elastic#180407 (comment). Specifically, we return back the validation errors to the modal window. An example of this modal is in the ticket description. - Additionally, on the Rule Editing page and **only for prebuilt rules** we: 1) hide the callout that says "You have an invalid input in this tab: ...", and 2) we don't show the modal if there are any data validation errors. We shouldn't show this modal and this callout until we release the prebuilt rule customization feature. 3) We will only validate the Actions tab. - Fix MKI flaky cypress tests introduced in elastic#191487 ([1](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/b1f442af-db44-8029-a9fb-7e3d988303b3?branch=main), [2](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/995655b6-ae70-86fd-b483-c65846cd8d66?branch=main), [3](https://buildkite.com/organizations/elastic/analytics/suites/serverless-mki-cypress-detection-engine/tests/02318f5c-6ca1-8779-a5a4-60f52a55b344?branch=main)). All three tests are failing due to missing `[data-test-subj="eqlRuleType"]` element. After checking and comparing my tests to other similar tests in the file, the only difference that I've found was extra `login();` call. Thus removing those. Here is the screen recording showing the new behaviour for prebuilt rules. The has missing data source query validation error, though we do not show it and allow user just to save the rule. Only Actions tab is validated on rule save action. https://github.com/user-attachments/assets/ce968f51-1a53-41b2-ad06-1b31dec085a6 ### Checklist Delete any items that are not applicable to this PR. - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed * [Detection Engine - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6925) (100 ESS & 100 Serverless) * [Rule Management - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6926) (100 ESS & 100 Serverless) * [Prebuilt Rules - Cypress](https://buildkite.com/elastic/kibana-flaky-test-suite-runner/builds/6927) (100 ESS & 100 Serverless) (cherry picked from commit c937e95)

…t rule editing (#191832) (#192683) (#192819) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Add validation error description on prebuilt rule editing (#191832) (#192683)](#192683)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Ievgen Sorokopud <[email protected]>

…e Editing page (#199550) **Resolves: #180172 ## Summary > [!NOTE] > Feature is behind the `prebuiltRulesCustomizationEnabled` feature flag. Removes the logic gates preventing prebuilt rules from being edited via the Rule Edit page behind the `prebuiltRulesCustomizationEnabled` feature flag. This allows all rules types to be fully editable via the UI. Also removes the muting logic we had in place for `Definition` tab warnings ([implemented here](#191487)) ### Screenshots #### _Before_ **Prebuilt rule only has the "Actions" tab enabled, users cannot customize anything else in the form** ![Screenshot 2024-11-08 at 3 08 15 PM](https://github.com/user-attachments/assets/b83836e6-f78f-4b3a-9fbc-55a5208250dd) #### _After_ **Prebuilt rule now has all tabs/fields available for editing and rule info is populated into the form** ![Screenshot 2024-11-08 at 3 02 43 PM](https://github.com/user-attachments/assets/184f6fc4-b64c-4e20-a987-76e460c61786) ### Checklist Delete any items that are not applicable to this PR. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) - [ ] This will appear in the **Release Notes** and follow the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <[email protected]>

…e Editing page (elastic#199550) **Resolves: elastic#180172 ## Summary > [!NOTE] > Feature is behind the `prebuiltRulesCustomizationEnabled` feature flag. Removes the logic gates preventing prebuilt rules from being edited via the Rule Edit page behind the `prebuiltRulesCustomizationEnabled` feature flag. This allows all rules types to be fully editable via the UI. Also removes the muting logic we had in place for `Definition` tab warnings ([implemented here](elastic#191487)) ### Screenshots #### _Before_ **Prebuilt rule only has the "Actions" tab enabled, users cannot customize anything else in the form** ![Screenshot 2024-11-08 at 3 08 15 PM](https://github.com/user-attachments/assets/b83836e6-f78f-4b3a-9fbc-55a5208250dd) #### _After_ **Prebuilt rule now has all tabs/fields available for editing and rule info is populated into the form** ![Screenshot 2024-11-08 at 3 02 43 PM](https://github.com/user-attachments/assets/184f6fc4-b64c-4e20-a987-76e460c61786) ### Checklist Delete any items that are not applicable to this PR. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) - [ ] This will appear in the **Release Notes** and follow the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit d6e6145)

…e Editing page (elastic#199550) **Resolves: elastic#180172 ## Summary > [!NOTE] > Feature is behind the `prebuiltRulesCustomizationEnabled` feature flag. Removes the logic gates preventing prebuilt rules from being edited via the Rule Edit page behind the `prebuiltRulesCustomizationEnabled` feature flag. This allows all rules types to be fully editable via the UI. Also removes the muting logic we had in place for `Definition` tab warnings ([implemented here](elastic#191487)) ### Screenshots #### _Before_ **Prebuilt rule only has the "Actions" tab enabled, users cannot customize anything else in the form** ![Screenshot 2024-11-08 at 3 08 15 PM](https://github.com/user-attachments/assets/b83836e6-f78f-4b3a-9fbc-55a5208250dd) #### _After_ **Prebuilt rule now has all tabs/fields available for editing and rule info is populated into the form** ![Screenshot 2024-11-08 at 3 02 43 PM](https://github.com/user-attachments/assets/184f6fc4-b64c-4e20-a987-76e460c61786) ### Checklist Delete any items that are not applicable to this PR. - [ ] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios ### For maintainers - [ ] This was checked for breaking API changes and was [labeled appropriately](https://www.elastic.co/guide/en/kibana/master/contributing.html#_add_your_labels) - [ ] This will appear in the **Release Notes** and follow the [guidelines](https://www.elastic.co/guide/en/kibana/master/contributing.html#kibana-release-notes-process) --------- Co-authored-by: Elastic Machine <[email protected]>

[Security Solution] Editing rules independently of source data (elast…

010088d

…ic#180407)

e40pud self-assigned this Aug 27, 2024

Fix tests

4ee535f

Merge branch 'main' into security/feature/180407-save-rules

d19001b

e40pud mentioned this pull request Aug 27, 2024

[Security Solution] Editing rules independently of source data #180407

Open

[CI] Auto-commit changed files from 'node scripts/eslint --no-cache -…

7b2616a

…-fix'

e40pud requested review from jpdjere and vitaliidm August 27, 2024 18:07

e40pud marked this pull request as ready for review August 27, 2024 18:07

e40pud requested a review from a team as a code owner August 27, 2024 18:07

elasticmachine and others added 2 commits August 28, 2024 11:53

Merge branch 'main' into security/feature/180407-save-rules

94e879b

Simplify confirmation modal description

b907087

elastic deleted a comment from kibanamachine Aug 29, 2024

vitaliidm reviewed Aug 29, 2024

View reviewed changes

Update x-pack/plugins/security_solution/public/detection_engine/rule_…

a76369e

…creation_ui/pages/translations.ts Co-authored-by: Vitalii Dmyterko <[email protected]>

Merge branch 'main' into security/feature/180407-save-rules

07a60cf

elasticmachine and others added 2 commits August 29, 2024 15:34

Merge branch 'main' into security/feature/180407-save-rules

3c42ada

Review feedback

2cfbf72

- memoize callbacks - clarify multiple EQL validation errors functionality - comments to clarify step editing button visibility conditions - add unit tests for `useRuleFormsErrors`

e40pud requested a review from a team as a code owner August 29, 2024 14:35

elastic deleted a comment from kibanamachine Aug 29, 2024

vitaliidm self-requested a review August 30, 2024 09:53

vitaliidm approved these changes Aug 30, 2024

View reviewed changes

e40pud mentioned this pull request Aug 30, 2024

[Security Solution] Add validation error description on prebuilt rule editing #191832

Open

e40pud merged commit 32fb434 into elastic:main Aug 30, 2024
43 checks passed

kibanamachine added v8.16.0 backport:skip This commit does not require backporting labels Aug 30, 2024

e40pud mentioned this pull request Sep 12, 2024

[Security Solution] Add validation error description on prebuilt rule editing (#191832) #192683

Merged

1 task

dplumlee mentioned this pull request Sep 24, 2024

[Security Solution] Add support for editing prebuilt rules to the Rule Editing page #180172

Closed

3 tasks

dplumlee mentioned this pull request Nov 8, 2024

[Security Solution] Add support for editing prebuilt rules to the Rule Editing page #199550

Merged

3 tasks

vitaliidm mentioned this pull request Nov 20, 2024

[Security Solution] Add ES|QL Query editable component #199887

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution] Editing rules independently of source data (#180407) #191487

[Security Solution] Editing rules independently of source data (#180407) #191487

e40pud commented Aug 27, 2024 •

edited

Loading

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

elasticmachine commented Aug 27, 2024

elasticmachine commented Aug 27, 2024

e40pud commented Aug 28, 2024

vitaliidm left a comment

vitaliidm Aug 29, 2024

e40pud Aug 29, 2024

vitaliidm Aug 29, 2024

e40pud Aug 29, 2024 •

edited

Loading

vitaliidm Aug 30, 2024

e40pud Aug 30, 2024

e40pud commented Aug 29, 2024

e40pud commented Aug 29, 2024

kibana-ci commented Aug 29, 2024 •

edited

Loading

kibanamachine commented Aug 30, 2024

kibanamachine commented Aug 30, 2024

kibanamachine commented Aug 30, 2024

[Security Solution] Editing rules independently of source data (#180407) #191487

[Security Solution] Editing rules independently of source data (#180407) #191487

Conversation

e40pud commented Aug 27, 2024 • edited Loading

Summary

Screenshot of the EQL rule creation workflow with existing non-blocking validation errors:

Screenshot of the EQL rule updating workflow with existing non-blocking validation errors:

UPDATE

Some test cases for local testing

Steps:

Steps:

Steps:

Steps:

Steps:

Steps:

Checklist

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

e40pud commented Aug 27, 2024

elasticmachine commented Aug 27, 2024

elasticmachine commented Aug 27, 2024

e40pud commented Aug 28, 2024

vitaliidm left a comment

Choose a reason for hiding this comment

vitaliidm Aug 29, 2024

Choose a reason for hiding this comment

e40pud Aug 29, 2024

Choose a reason for hiding this comment

vitaliidm Aug 29, 2024

Choose a reason for hiding this comment

e40pud Aug 29, 2024 • edited Loading

Choose a reason for hiding this comment

vitaliidm Aug 30, 2024

Choose a reason for hiding this comment

e40pud Aug 30, 2024

Choose a reason for hiding this comment

e40pud commented Aug 29, 2024

e40pud commented Aug 29, 2024

kibana-ci commented Aug 29, 2024 • edited Loading

💛 Build succeeded, but was flaky

Failed CI Steps

Metrics [docs]

Module Count

Async chunks

History

kibanamachine commented Aug 30, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6830

kibanamachine commented Aug 30, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6828

kibanamachine commented Aug 30, 2024

Flaky Test Runner Stats

🟠 Some tests failed. - kibana-flaky-test-suite-runner#6831

e40pud commented Aug 27, 2024 •

edited

Loading

e40pud Aug 29, 2024 •

edited

Loading

kibana-ci commented Aug 29, 2024 •

edited

Loading