[Security Solution] Add ES|QL Query editable component #199887
Merged
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
maximpn
added
release_note:skip
|
🤖 Jobs for this PR can be triggered through checkboxes. 🚧
ℹ️ To trigger the CI, please tick the checkbox below 👇
|
maximpn
force-pushed
the
add-esql-query-editable-component
branch
3 times, most recently
from
3c63ea7 to
5a0566c
Compare
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
maximpn
requested review from
nikitaindik and
vitaliidm
and removed request for
rylnd and
xcrzx
69 tasks
|
Probably not related to this PR, but the help text doesn't seem to be correct here:
|
|
Kibana crashes when I try to open the upgrade flyout for this specific rule:
|
|
Hi @xcrzx, are unrelated to this PR
|
Do you have a list of issues that will be addressed separately? |
The ticket #171520 has tasks (under |
|
Not directly related to this PR, the diff view highlights changes incorrectly:
Removed parts (in bold) are in the wrong places |
|
Auto-merge is also behaving incorrectly for the query fields. Incoming changes are simply ignored, and the current version is proposed as the target. For context:
There’s no actual conflict since different lines were touched, yet the merge does not account for the incoming changes.
The correctly merged version would look like this:
|
|
Not sure if the editor itself is broken or there's something related to its integration, but when I try to go to a new line (by hitting Enter), random symbols are inserted instead: Screen.Recording.2024-11-27.at.13.22.58.mov |
xcrzx
reviewed
Nov 27, 2024
...rule_creation/components/esql_query_edit/validators/esql_query_required_validator_factory.ts
Outdated
Show resolved
Hide resolved
...rule_creation/components/esql_query_edit/validators/esql_query_required_validator_factory.ts
Outdated
Show resolved
Hide resolved
.../plugins/security_solution/public/detection_engine/rule_creation/logic/esql_query_columns.ts
Show resolved
Hide resolved
.../plugins/security_solution/public/detection_engine/rule_creation/logic/esql_query_columns.ts
Show resolved
Hide resolved
...ine/rule_creation/components/esql_query_edit/validators/esql_query_validator_factory.test.ts
Outdated
Show resolved
Hide resolved
|
Thanks for review. I addressed you last comments. Issue you found in #199887 (comment), #199887 (comment) and #199887 (comment) lay out of scope of this PR. Let's address them separately. |
|
The recent changes in ES|QL query columns fetching, when errors are not handled, pollute dev console. Every input error is displaying there now Screen.Recording.2024-11-28.at.10.01.54.mov |
Yes. It's a known drawback. We need to fetch ES|QL columns for the other input via |
|
The nature of ES|QL API is that it returns error on invalid input from user. It includes syntax error, not fully finished query or missing index/data on server side(query is valid from syntax wise). That's why we had to catch error from response and process it, depends on our case:
Now, all error go through. |
maximpn
force-pushed
the
add-esql-query-editable-component
branch
from
08d36c7 to
625ffa9
Compare
|
I rolled back the last commit. |
|
Starting backport for target branches: 8.x |
💚 Build Succeeded
Metrics [docs]Module Count
Async chunks
Unknown metric groupsReferences to deprecated APIs
History
cc @maximpn |
💔 All backports failed
Manual backportTo create the backport manually run: Questions ?Please refer to the Backport tool documentation |
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
maximpn
added a commit
to maximpn/kibana
that referenced
this pull request
Nov 28, 2024
**Partially addresses:** elastic#171520 ## Summary This PR adds is built on top of elastic#193828 and elastic#196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow. ## Details This PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made - ES|QL validator was refactored and covered by unit tests - Query persistence was addressed and covered by tests (previous functionality didn't work out of the box and didn't have tests) ## How to test The simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Run Kibana locally - Install an ES|QL prebuilt rule, e.g. `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` with rule_id `0cd2f3e6-41da-40e6-b28b-466f688f00a6` - Patch the installed rule by running a query below ```bash curl -X PATCH --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"rule_id":"0cd2f3e6-41da-40e6-b28b-466f688f00a6","version":1,"query":"from logs-*","language":"esql"}' http://localhost:5601/kbn/api/detection_engine/rules ``` - Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button ## Screenshots <img width="2550" alt="image" src="https://github.com/user-attachments/assets/f65d04d5-9fd9-4d3f-8741-eba04a3be8a6"> <img width="2552" alt="image" src="https://github.com/user-attachments/assets/dd0a2613-5262-44b2-bbeb-d0ed34d57d9c"> (cherry picked from commit e55232f) # Conflicts: # x-pack/plugins/security_solution/public/detection_engine/rule_creation/logic/esql_validator.test.ts # x-pack/plugins/security_solution/public/detection_engine/rule_creation/logic/esql_validator.ts # x-pack/plugins/translations/translations/zh-CN.json
maximpn
added a commit
that referenced
this pull request
Nov 29, 2024
#202224) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Add ES|QL Query editable component (#199887)](#199887) <!--- Backport version: 8.9.8 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Maxim Palenov","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-28T14:48:48Z","message":"[Security Solution] Add ES|QL Query editable component (#199887)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/171520\r\n\r\n## Summary\r\n\r\nThis PR adds is built on top of #193828 and #196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow.\r\n\r\n## Details\r\n\r\nThis PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made\r\n\r\n- ES|QL validator was refactored and covered by unit tests\r\n- Query persistence was addressed and covered by tests (previous functionality didn't work out of the box and didn't have tests)\r\n\r\n## How to test\r\n\r\nThe simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled\r\n- Run Kibana locally\r\n- Install an ES|QL prebuilt rule, e.g. `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` with rule_id `0cd2f3e6-41da-40e6-b28b-466f688f00a6`\r\n- Patch the installed rule by running a query below\r\n\r\n```bash\r\ncurl -X PATCH --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d '{\"rule_id\":\"0cd2f3e6-41da-40e6-b28b-466f688f00a6\",\"version\":1,\"query\":\"from logs-*\",\"language\":\"esql\"}' http://localhost:5601/kbn/api/detection_engine/rules\r\n```\r\n\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button\r\n\r\n## Screenshots\r\n\r\n<img width=\"2550\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f65d04d5-9fd9-4d3f-8741-eba04a3be8a6\">\r\n\r\n<img width=\"2552\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dd0a2613-5262-44b2-bbeb-d0ed34d57d9c\">","sha":"e55232f87732594eb9502993766203a219be44db","branchLabelMapping":{"^v9.0.0$":"main","^v8.18.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.18.0"],"number":199887,"url":"https://github.com/elastic/kibana/pull/199887","mergeCommit":{"message":"[Security Solution] Add ES|QL Query editable component (#199887)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/171520\r\n\r\n## Summary\r\n\r\nThis PR adds is built on top of #193828 and #196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow.\r\n\r\n## Details\r\n\r\nThis PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made\r\n\r\n- ES|QL validator was refactored and covered by unit tests\r\n- Query persistence was addressed and covered by tests (previous functionality didn't work out of the box and didn't have tests)\r\n\r\n## How to test\r\n\r\nThe simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled\r\n- Run Kibana locally\r\n- Install an ES|QL prebuilt rule, e.g. `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` with rule_id `0cd2f3e6-41da-40e6-b28b-466f688f00a6`\r\n- Patch the installed rule by running a query below\r\n\r\n```bash\r\ncurl -X PATCH --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d '{\"rule_id\":\"0cd2f3e6-41da-40e6-b28b-466f688f00a6\",\"version\":1,\"query\":\"from logs-*\",\"language\":\"esql\"}' http://localhost:5601/kbn/api/detection_engine/rules\r\n```\r\n\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button\r\n\r\n## Screenshots\r\n\r\n<img width=\"2550\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f65d04d5-9fd9-4d3f-8741-eba04a3be8a6\">\r\n\r\n<img width=\"2552\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dd0a2613-5262-44b2-bbeb-d0ed34d57d9c\">","sha":"e55232f87732594eb9502993766203a219be44db"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","labelRegex":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/199887","number":199887,"mergeCommit":{"message":"[Security Solution] Add ES|QL Query editable component (#199887)\n\n**Partially addresses:** https://github.com/elastic/kibana/issues/171520\r\n\r\n## Summary\r\n\r\nThis PR adds is built on top of #193828 and #196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow.\r\n\r\n## Details\r\n\r\nThis PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made\r\n\r\n- ES|QL validator was refactored and covered by unit tests\r\n- Query persistence was addressed and covered by tests (previous functionality didn't work out of the box and didn't have tests)\r\n\r\n## How to test\r\n\r\nThe simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below\r\n\r\n- Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled\r\n- Run Kibana locally\r\n- Install an ES|QL prebuilt rule, e.g. `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` with rule_id `0cd2f3e6-41da-40e6-b28b-466f688f00a6`\r\n- Patch the installed rule by running a query below\r\n\r\n```bash\r\ncurl -X PATCH --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H \"elastic-api-version: 2023-10-31\" -d '{\"rule_id\":\"0cd2f3e6-41da-40e6-b28b-466f688f00a6\",\"version\":1,\"query\":\"from logs-*\",\"language\":\"esql\"}' http://localhost:5601/kbn/api/detection_engine/rules\r\n```\r\n\r\n- Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button\r\n\r\n## Screenshots\r\n\r\n<img width=\"2550\" alt=\"image\" src=\"https://github.com/user-attachments/assets/f65d04d5-9fd9-4d3f-8741-eba04a3be8a6\">\r\n\r\n<img width=\"2552\" alt=\"image\" src=\"https://github.com/user-attachments/assets/dd0a2613-5262-44b2-bbeb-d0ed34d57d9c\">","sha":"e55232f87732594eb9502993766203a219be44db"}},{"branch":"8.x","label":"v8.18.0","labelRegex":"^v8.18.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT-->
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Dec 12, 2024
**Partially addresses:** elastic#171520 ## Summary This PR adds is built on top of elastic#193828 and elastic#196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow. ## Details This PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made - ES|QL validator was refactored and covered by unit tests - Query persistence was addressed and covered by tests (previous functionality didn't work out of the box and didn't have tests) ## How to test The simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below - Ensure the `prebuiltRulesCustomizationEnabled` feature flag is enabled - Run Kibana locally - Install an ES|QL prebuilt rule, e.g. `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` with rule_id `0cd2f3e6-41da-40e6-b28b-466f688f00a6` - Patch the installed rule by running a query below ```bash curl -X PATCH --user elastic:changeme -H 'Content-Type: application/json' -H 'kbn-xsrf: 123' -H "elastic-api-version: 2023-10-31" -d '{"rule_id":"0cd2f3e6-41da-40e6-b28b-466f688f00a6","version":1,"query":"from logs-*","language":"esql"}' http://localhost:5601/kbn/api/detection_engine/rules ``` - Open `Detection Rules (SIEM)` Page -> `Rule Updates` -> click on `AWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Session` rule -> expand `EQL Query` to see EQL Query -> press `Edit` button ## Screenshots <img width="2550" alt="image" src="https://github.com/user-attachments/assets/f65d04d5-9fd9-4d3f-8741-eba04a3be8a6"> <img width="2552" alt="image" src="https://github.com/user-attachments/assets/dd0a2613-5262-44b2-bbeb-d0ed34d57d9c">
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Partially addresses: #171520
Summary
This PR adds is built on top of #193828 and #196948 and adds an ES|QL Query editable component for Three Way Diff tab's final edit side of the upgrade prebuilt rule workflow.
Details
This PR extracts ES|QL Query edit component from Define rule form step and makes it reusable. The following changes were made
How to test
The simplest way to test is via patching installed prebuilt rules (a.k.a. downgrading a prebuilt rule) via Rule Patch API. Please follow steps below
prebuiltRulesCustomizationEnabledfeature flag is enabledAWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Sessionwith rule_id0cd2f3e6-41da-40e6-b28b-466f688f00a6Detection Rules (SIEM)Page ->Rule Updates-> click onAWS Bedrock Guardrails Detected Multiple Violations by a Single User Over a Sessionrule -> expandEQL Queryto see EQL Query -> pressEditbuttonScreenshots