[Security Solution] Test plans for prebuilt rule import and export #191116
+192
−0
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
rylnd
added
test-plan
Team:Detection Rule Management
4 tasks
rylnd
added a commit
that referenced
this pull request
Oct 16, 2024
…90198) ## Summary This PR introduces the backend functionality necessary to import prebuilt rules via our existing import API. The [RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md) goes into detail, and the import-specific issue is described [here](#180168), but at a high level we're adding two things in this PR: 1. The logic to calculate the new `rule_source` field on import, which contains the information about the rule's relationship to existing prebuilt rules. 2. A new code path for importing rules, which respects the calculated `rule_source` field. In order to maintain backwards compatibility with the existing import logic, and because the existing import implementation is hard to modify/extend, I opted to add a top-level switch on the feature flag in the import route itself, which calls either the existing import function (now named `importRulesLegacy`), or the new function, `importRules`, which ultimately calls the new `DetectionRulesClient` method, `importRules`. Neither knows about the feature flag, and thanks to great suggestions from @xcrzx there are nice, clean boundaries between the import functions and the client methods. I went down the path of trying to write the new import code by reusing the outer `importRules` function, but after discussion with the team we decided it was simplest to simply bifurcate the code at that point, so that we have: 1. The legacy import code, which: * only supports custom rules (`immutable: false`) * accepts `version` as an optional parameter * calculates a legacy `rule_source` value based on the `immutable` field 2. The new import code, which * Installs the prebuilt rules assets as necessary * Accepts both kinds of rules (`immutable: true/false`) * Requires `version` as a parameter for _prebuilt_ rules * Allows `version` to be optional for custom rules * calculates a proper `rule_source` (and `immutable`) ### Deprecation of `immutable` The RFC (and thus I) had initially thought that we'd be deprecating the `immutable` field as part of this PR/Epic. However, after [discussion](#190198 (comment)) we have opted to continue supporting `immutable` until such time as we can drop it, compatibility-wise. ## Steps to Review 1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled` 2. Install the prebuilt rules package via fleet 3. Create and export a custom rule to serve as a template (or download one: curl -L -H 'Authorization: 8eef0fe5d7dfc52b' -o 'rules_export (1).ndjson' https://upload.elastic.co/d/4693e7fe4356ce7bcf7b7d6b72881a9fd189730c61bf5ef47c9930458d746979 ) 4. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`, e.g. `ac8805f6-1e08-406c-962e-3937057fa86f` 5. Edit the `rules_export.ndjson` to contain only the first line, and modify the `rule_id` with the prebuilt rule's 6. Import `rules_export.ndjson` and then retrieve the rule via the Dev Console: GET kbn:api/detection_engine/rules?rule_id=ac8805f6-1e08-406c-962e-3937057fa86f 7. Observe that the rule has the expected `rule_source` and `immutable` values 8. Test other permutations of import by modifying `rules_export.ndjson` and re-importing; see ([the test plan](#191116) as well as a [reference table of scenarios](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md#importing-rules)) ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Georgii Gorbachev <[email protected]>
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Oct 16, 2024
…astic#190198) ## Summary This PR introduces the backend functionality necessary to import prebuilt rules via our existing import API. The [RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md) goes into detail, and the import-specific issue is described [here](elastic#180168), but at a high level we're adding two things in this PR: 1. The logic to calculate the new `rule_source` field on import, which contains the information about the rule's relationship to existing prebuilt rules. 2. A new code path for importing rules, which respects the calculated `rule_source` field. In order to maintain backwards compatibility with the existing import logic, and because the existing import implementation is hard to modify/extend, I opted to add a top-level switch on the feature flag in the import route itself, which calls either the existing import function (now named `importRulesLegacy`), or the new function, `importRules`, which ultimately calls the new `DetectionRulesClient` method, `importRules`. Neither knows about the feature flag, and thanks to great suggestions from @xcrzx there are nice, clean boundaries between the import functions and the client methods. I went down the path of trying to write the new import code by reusing the outer `importRules` function, but after discussion with the team we decided it was simplest to simply bifurcate the code at that point, so that we have: 1. The legacy import code, which: * only supports custom rules (`immutable: false`) * accepts `version` as an optional parameter * calculates a legacy `rule_source` value based on the `immutable` field 2. The new import code, which * Installs the prebuilt rules assets as necessary * Accepts both kinds of rules (`immutable: true/false`) * Requires `version` as a parameter for _prebuilt_ rules * Allows `version` to be optional for custom rules * calculates a proper `rule_source` (and `immutable`) ### Deprecation of `immutable` The RFC (and thus I) had initially thought that we'd be deprecating the `immutable` field as part of this PR/Epic. However, after [discussion](elastic#190198 (comment)) we have opted to continue supporting `immutable` until such time as we can drop it, compatibility-wise. ## Steps to Review 1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled` 2. Install the prebuilt rules package via fleet 3. Create and export a custom rule to serve as a template (or download one: curl -L -H 'Authorization: 8eef0fe5d7dfc52b' -o 'rules_export (1).ndjson' https://upload.elastic.co/d/4693e7fe4356ce7bcf7b7d6b72881a9fd189730c61bf5ef47c9930458d746979 ) 4. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`, e.g. `ac8805f6-1e08-406c-962e-3937057fa86f` 5. Edit the `rules_export.ndjson` to contain only the first line, and modify the `rule_id` with the prebuilt rule's 6. Import `rules_export.ndjson` and then retrieve the rule via the Dev Console: GET kbn:api/detection_engine/rules?rule_id=ac8805f6-1e08-406c-962e-3937057fa86f 7. Observe that the rule has the expected `rule_source` and `immutable` values 8. Test other permutations of import by modifying `rules_export.ndjson` and re-importing; see ([the test plan](elastic#191116) as well as a [reference table of scenarios](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md#importing-rules)) ### Checklist Delete any items that are not applicable to this PR. - [x] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md) - [ ] [Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html) was added for features that require explanation or tutorials - [x] [Unit or functional tests](https://www.elastic.co/guide/en/kibana/master/development-tests.html) were updated or added to match the most common scenarios - [ ] [Flaky Test Runner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was used on any tests changed --------- Co-authored-by: kibanamachine <[email protected]> Co-authored-by: Georgii Gorbachev <[email protected]> (cherry picked from commit c0b1301)
kibanamachine
added a commit
that referenced
this pull request
Oct 16, 2024
…PI (#190198) (#196613) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Allow importing of prebuilt rules via the API (#190198)](#190198) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Ryland Herrick","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-10-16T21:19:05Z","message":"[Security Solution] Allow importing of prebuilt rules via the API (#190198)\n\n## Summary\r\n\r\nThis PR introduces the backend functionality necessary to import\r\nprebuilt rules via our existing import API. The\r\n[RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md)\r\ngoes into detail, and the import-specific issue is described\r\n[here](#180168), but at a high\r\nlevel we're adding two things in this PR:\r\n\r\n1. The logic to calculate the new `rule_source` field on import, which\r\ncontains the information about the rule's relationship to existing\r\nprebuilt rules.\r\n2. A new code path for importing rules, which respects the calculated\r\n`rule_source` field.\r\n\r\nIn order to maintain backwards compatibility with the existing import\r\nlogic, and because the existing import implementation is hard to\r\nmodify/extend, I opted to add a top-level switch on the feature flag in\r\nthe import route itself, which calls either the existing import function\r\n(now named `importRulesLegacy`), or the new function, `importRules`,\r\nwhich ultimately calls the new `DetectionRulesClient` method,\r\n`importRules`. Neither knows about the feature flag, and thanks to great\r\nsuggestions from @xcrzx there are nice, clean boundaries between the\r\nimport functions and the client methods.\r\n\r\nI went down the path of trying to write the new import code by reusing\r\nthe outer `importRules` function, but after discussion with the team we\r\ndecided it was simplest to simply bifurcate the code at that point, so\r\nthat we have:\r\n\r\n1. The legacy import code, which:\r\n * only supports custom rules (`immutable: false`)\r\n * accepts `version` as an optional parameter\r\n* calculates a legacy `rule_source` value based on the `immutable` field\r\n2. The new import code, which\r\n * Installs the prebuilt rules assets as necessary\r\n * Accepts both kinds of rules (`immutable: true/false`)\r\n * Requires `version` as a parameter for _prebuilt_ rules\r\n * Allows `version` to be optional for custom rules\r\n * calculates a proper `rule_source` (and `immutable`)\r\n\r\n\r\n### Deprecation of `immutable`\r\nThe RFC (and thus I) had initially thought that we'd be deprecating the\r\n`immutable` field as part of this PR/Epic. However, after\r\n[discussion](https://github.com/elastic/kibana/pull/190198#discussion_r1736021749)\r\nwe have opted to continue supporting `immutable` until such time as we\r\ncan drop it, compatibility-wise.\r\n\r\n\r\n## Steps to Review\r\n1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled`\r\n2. Install the prebuilt rules package via fleet\r\n3. Create and export a custom rule to serve as a template (or download\r\none:\r\n\r\ncurl -L -H 'Authorization: 8eef0fe5d7dfc52b' -o 'rules_export\r\n(1).ndjson'\r\nhttps://upload.elastic.co/d/4693e7fe4356ce7bcf7b7d6b72881a9fd189730c61bf5ef47c9930458d746979\r\n )\r\n \r\n4. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`,\r\ne.g. `ac8805f6-1e08-406c-962e-3937057fa86f`\r\n5. Edit the `rules_export.ndjson` to contain only the first line, and\r\nmodify the `rule_id` with the prebuilt rule's\r\n6. Import `rules_export.ndjson` and then retrieve the rule via the Dev\r\nConsole:\r\n\r\nGET\r\nkbn:api/detection_engine/rules?rule_id=ac8805f6-1e08-406c-962e-3937057fa86f\r\n\r\n7. Observe that the rule has the expected `rule_source` and `immutable`\r\nvalues\r\n8. Test other permutations of import by modifying `rules_export.ndjson`\r\nand re-importing; see ([the test\r\nplan](#191116) as well as a\r\n[reference table of\r\nscenarios](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md#importing-rules))\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Georgii Gorbachev <[email protected]>","sha":"c0b1301a21b74e8b643fc0e898b1d982fd85d98e","branchLabelMapping":{"^v9.0.0$":"main","^v8.16.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","v8.16.0","backport:version"],"title":"[Security Solution] Allow importing of prebuilt rules via the API","number":190198,"url":"https://github.com/elastic/kibana/pull/190198","mergeCommit":{"message":"[Security Solution] Allow importing of prebuilt rules via the API (#190198)\n\n## Summary\r\n\r\nThis PR introduces the backend functionality necessary to import\r\nprebuilt rules via our existing import API. The\r\n[RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md)\r\ngoes into detail, and the import-specific issue is described\r\n[here](#180168), but at a high\r\nlevel we're adding two things in this PR:\r\n\r\n1. The logic to calculate the new `rule_source` field on import, which\r\ncontains the information about the rule's relationship to existing\r\nprebuilt rules.\r\n2. A new code path for importing rules, which respects the calculated\r\n`rule_source` field.\r\n\r\nIn order to maintain backwards compatibility with the existing import\r\nlogic, and because the existing import implementation is hard to\r\nmodify/extend, I opted to add a top-level switch on the feature flag in\r\nthe import route itself, which calls either the existing import function\r\n(now named `importRulesLegacy`), or the new function, `importRules`,\r\nwhich ultimately calls the new `DetectionRulesClient` method,\r\n`importRules`. Neither knows about the feature flag, and thanks to great\r\nsuggestions from @xcrzx there are nice, clean boundaries between the\r\nimport functions and the client methods.\r\n\r\nI went down the path of trying to write the new import code by reusing\r\nthe outer `importRules` function, but after discussion with the team we\r\ndecided it was simplest to simply bifurcate the code at that point, so\r\nthat we have:\r\n\r\n1. The legacy import code, which:\r\n * only supports custom rules (`immutable: false`)\r\n * accepts `version` as an optional parameter\r\n* calculates a legacy `rule_source` value based on the `immutable` field\r\n2. The new import code, which\r\n * Installs the prebuilt rules assets as necessary\r\n * Accepts both kinds of rules (`immutable: true/false`)\r\n * Requires `version` as a parameter for _prebuilt_ rules\r\n * Allows `version` to be optional for custom rules\r\n * calculates a proper `rule_source` (and `immutable`)\r\n\r\n\r\n### Deprecation of `immutable`\r\nThe RFC (and thus I) had initially thought that we'd be deprecating the\r\n`immutable` field as part of this PR/Epic. However, after\r\n[discussion](https://github.com/elastic/kibana/pull/190198#discussion_r1736021749)\r\nwe have opted to continue supporting `immutable` until such time as we\r\ncan drop it, compatibility-wise.\r\n\r\n\r\n## Steps to Review\r\n1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled`\r\n2. Install the prebuilt rules package via fleet\r\n3. Create and export a custom rule to serve as a template (or download\r\none:\r\n\r\ncurl -L -H 'Authorization: 8eef0fe5d7dfc52b' -o 'rules_export\r\n(1).ndjson'\r\nhttps://upload.elastic.co/d/4693e7fe4356ce7bcf7b7d6b72881a9fd189730c61bf5ef47c9930458d746979\r\n )\r\n \r\n4. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`,\r\ne.g. `ac8805f6-1e08-406c-962e-3937057fa86f`\r\n5. Edit the `rules_export.ndjson` to contain only the first line, and\r\nmodify the `rule_id` with the prebuilt rule's\r\n6. Import `rules_export.ndjson` and then retrieve the rule via the Dev\r\nConsole:\r\n\r\nGET\r\nkbn:api/detection_engine/rules?rule_id=ac8805f6-1e08-406c-962e-3937057fa86f\r\n\r\n7. Observe that the rule has the expected `rule_source` and `immutable`\r\nvalues\r\n8. Test other permutations of import by modifying `rules_export.ndjson`\r\nand re-importing; see ([the test\r\nplan](#191116) as well as a\r\n[reference table of\r\nscenarios](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md#importing-rules))\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Georgii Gorbachev <[email protected]>","sha":"c0b1301a21b74e8b643fc0e898b1d982fd85d98e"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/190198","number":190198,"mergeCommit":{"message":"[Security Solution] Allow importing of prebuilt rules via the API (#190198)\n\n## Summary\r\n\r\nThis PR introduces the backend functionality necessary to import\r\nprebuilt rules via our existing import API. The\r\n[RFC](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md)\r\ngoes into detail, and the import-specific issue is described\r\n[here](#180168), but at a high\r\nlevel we're adding two things in this PR:\r\n\r\n1. The logic to calculate the new `rule_source` field on import, which\r\ncontains the information about the rule's relationship to existing\r\nprebuilt rules.\r\n2. A new code path for importing rules, which respects the calculated\r\n`rule_source` field.\r\n\r\nIn order to maintain backwards compatibility with the existing import\r\nlogic, and because the existing import implementation is hard to\r\nmodify/extend, I opted to add a top-level switch on the feature flag in\r\nthe import route itself, which calls either the existing import function\r\n(now named `importRulesLegacy`), or the new function, `importRules`,\r\nwhich ultimately calls the new `DetectionRulesClient` method,\r\n`importRules`. Neither knows about the feature flag, and thanks to great\r\nsuggestions from @xcrzx there are nice, clean boundaries between the\r\nimport functions and the client methods.\r\n\r\nI went down the path of trying to write the new import code by reusing\r\nthe outer `importRules` function, but after discussion with the team we\r\ndecided it was simplest to simply bifurcate the code at that point, so\r\nthat we have:\r\n\r\n1. The legacy import code, which:\r\n * only supports custom rules (`immutable: false`)\r\n * accepts `version` as an optional parameter\r\n* calculates a legacy `rule_source` value based on the `immutable` field\r\n2. The new import code, which\r\n * Installs the prebuilt rules assets as necessary\r\n * Accepts both kinds of rules (`immutable: true/false`)\r\n * Requires `version` as a parameter for _prebuilt_ rules\r\n * Allows `version` to be optional for custom rules\r\n * calculates a proper `rule_source` (and `immutable`)\r\n\r\n\r\n### Deprecation of `immutable`\r\nThe RFC (and thus I) had initially thought that we'd be deprecating the\r\n`immutable` field as part of this PR/Epic. However, after\r\n[discussion](https://github.com/elastic/kibana/pull/190198#discussion_r1736021749)\r\nwe have opted to continue supporting `immutable` until such time as we\r\ncan drop it, compatibility-wise.\r\n\r\n\r\n## Steps to Review\r\n1. Enable the Feature Flag: `prebuiltRulesCustomizationEnabled`\r\n2. Install the prebuilt rules package via fleet\r\n3. Create and export a custom rule to serve as a template (or download\r\none:\r\n\r\ncurl -L -H 'Authorization: 8eef0fe5d7dfc52b' -o 'rules_export\r\n(1).ndjson'\r\nhttps://upload.elastic.co/d/4693e7fe4356ce7bcf7b7d6b72881a9fd189730c61bf5ef47c9930458d746979\r\n )\r\n \r\n4. Install some prebuilt rules, and obtain a prebuilt rule's `rule_id`,\r\ne.g. `ac8805f6-1e08-406c-962e-3937057fa86f`\r\n5. Edit the `rules_export.ndjson` to contain only the first line, and\r\nmodify the `rule_id` with the prebuilt rule's\r\n6. Import `rules_export.ndjson` and then retrieve the rule via the Dev\r\nConsole:\r\n\r\nGET\r\nkbn:api/detection_engine/rules?rule_id=ac8805f6-1e08-406c-962e-3937057fa86f\r\n\r\n7. Observe that the rule has the expected `rule_source` and `immutable`\r\nvalues\r\n8. Test other permutations of import by modifying `rules_export.ndjson`\r\nand re-importing; see ([the test\r\nplan](#191116) as well as a\r\n[reference table of\r\nscenarios](https://github.com/elastic/kibana/blob/main/x-pack/plugins/security_solution/docs/rfcs/detection_response/prebuilt_rules_customization.md#importing-rules))\r\n\r\n### Checklist\r\n\r\nDelete any items that are not applicable to this PR.\r\n\r\n- [x] Any text added follows [EUI's writing\r\nguidelines](https://elastic.github.io/eui/#/guidelines/writing), uses\r\nsentence case text and includes [i18n\r\nsupport](https://github.com/elastic/kibana/blob/main/packages/kbn-i18n/README.md)\r\n- [ ]\r\n[Documentation](https://www.elastic.co/guide/en/kibana/master/development-documentation.html)\r\nwas added for features that require explanation or tutorials\r\n- [x] [Unit or functional\r\ntests](https://www.elastic.co/guide/en/kibana/master/development-tests.html)\r\nwere updated or added to match the most common scenarios\r\n- [ ] [Flaky Test\r\nRunner](https://ci-stats.kibana.dev/trigger_flaky_test_runner/1) was\r\nused on any tests changed\r\n\r\n---------\r\n\r\nCo-authored-by: kibanamachine <[email protected]>\r\nCo-authored-by: Georgii Gorbachev <[email protected]>","sha":"c0b1301a21b74e8b643fc0e898b1d982fd85d98e"}},{"branch":"8.x","label":"v8.16.0","branchLabelMappingKey":"^v8.16.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Ryland Herrick <[email protected]>
I'm not sure if these belong with an existing file, and much of the surrounding context is not there, currently.
When we can't track a rule down to a known version, we assume that it is customized by default.
With the main backend PRs having been merged, the logic and problem scope is now well-defined enough that I can confidently say these tests cover the correct scenarios.
These are notably simpler than import. I've included an error case, mainly for the purposes of discussion.
rylnd
force-pushed
the
ryland/prebuilt_rule_import_test_plan
branch
from
13cecbc to
b245c35
Compare
|
Pinging @elastic/security-detection-rule-management (Team:Detection Rule Management) |
💚 Build Succeeded
cc @rylnd |
rylnd
added
backport:skip
banderror
added
Team:Detections and Resp
|
Pinging @elastic/security-solution (Team: SecuritySolution) |
|
Pinging @elastic/security-detections-response (Team:Detections and Resp) |
|
Hi @pborgonovi @jpdjere, could you please review the new plans for prebuilt rules import and export? You might find the corresponding implementation PRs (links in the description) and the RFC useful to compare the plans with the actual behavior and original cases described in the RFC. |
We generally assume that the incoming rule represents the intended outcome. If it did not, why would it be imported? |
|
It's very clear. Thank you @rylnd |
pborgonovi
approved these changes
Oct 24, 2024
dplumlee
reviewed
Oct 28, 2024
There was a problem hiding this comment.
Reading back through the RFC and the related PRs, I think these test cases are well written and comprehensive, thanks @rylnd! I did have one comment about formatting, most of the time we've been following this template for test plans within rule management, it might be worth adding some of that markdown formatting just for consistency's sake - I'm not sure how strict we're wanting to be on that.
| Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version | ||
| Given the import payload contains a prebuilt rule with a matching rule_id and version, identical to the published rule | ||
| When the user imports the rule | ||
| Then the rule should be created or updated |
There was a problem hiding this comment.
Is it worth having the specificity between these two here? What causes the rule to be created vs updated?
There was a problem hiding this comment.
@dplumlee @pborgonovi brought up similar questions, which were discussed here, but the TL;DR:
When the test states Then the rule should be created or updated, that's really a shorthand for:
If the rule is new, create it
If the rule exists, update or reject based on the value of overwrite
But since that's all orthogonal to the prebuilt customization stuff, I didn't think it was an important thing to split out in these tests.
|
@elasticmachine merge upstream |
banderror
added
v9.0.0
backport:version
|
@dplumlee I revised the formatting, thanks for the note! |
dplumlee
approved these changes
Nov 9, 2024
|
Starting backport for target branches: 8.x |
kibanamachine
pushed a commit
to kibanamachine/kibana
that referenced
this pull request
Nov 11, 2024
…lastic#191116) ## Summary This PR introduces test plans for both [Prebuilt Rule Import](elastic#180168) (corresponding [PR](elastic#190198)) and [Prebuilt Rule Export](elastic#180167) (corresponding [PR](elastic#194498)). Import is considerably more complicated as it is calculating new values (for `rule_source`, `immutable`), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported). --------- Co-authored-by: Elastic Machine <[email protected]> (cherry picked from commit e429849)
💚 All backports created successfully
Note: Successful backport PRs will be merged automatically after passing CI. Questions ?Please refer to the Backport tool documentation |
kibanamachine
added a commit
that referenced
this pull request
Nov 11, 2024
…ort (#191116) (#199716) # Backport This will backport the following commits from `main` to `8.x`: - [[Security Solution] Test plans for prebuilt rule import and export (#191116)](#191116) <!--- Backport version: 9.4.3 --> ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport) <!--BACKPORT [{"author":{"name":"Ryland Herrick","email":"[email protected]"},"sourceCommit":{"committedDate":"2024-11-11T21:01:46Z","message":"[Security Solution] Test plans for prebuilt rule import and export (#191116)\n\n## Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt Rule\r\nImport](#180168) (corresponding\r\n[PR](#190198)) and [Prebuilt Rule\r\nExport](#180167) (corresponding\r\n[PR](#194498)). Import is\r\nconsiderably more complicated as it is calculating new values (for\r\n`rule_source`, `immutable`), while the export work is mainly removing\r\nexisting restrictions (which allowed only custom rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103","branchLabelMapping":{"^v9.0.0$":"main","^v8.17.0$":"8.x","^v(\\d+).(\\d+).\\d+$":"$1.$2"}},"sourcePullRequest":{"labels":["release_note:skip","test-plan","v9.0.0","Team:Detections and Resp","Team: SecuritySolution","Team:Detection Rule Management","Feature:Prebuilt Detection Rules","backport:version","v8.17.0"],"title":"[Security Solution] Test plans for prebuilt rule import and export","number":191116,"url":"https://github.com/elastic/kibana/pull/191116","mergeCommit":{"message":"[Security Solution] Test plans for prebuilt rule import and export (#191116)\n\n## Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt Rule\r\nImport](#180168) (corresponding\r\n[PR](#190198)) and [Prebuilt Rule\r\nExport](#180167) (corresponding\r\n[PR](#194498)). Import is\r\nconsiderably more complicated as it is calculating new values (for\r\n`rule_source`, `immutable`), while the export work is mainly removing\r\nexisting restrictions (which allowed only custom rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},"sourceBranch":"main","suggestedTargetBranches":["8.x"],"targetPullRequestStates":[{"branch":"main","label":"v9.0.0","branchLabelMappingKey":"^v9.0.0$","isSourceBranch":true,"state":"MERGED","url":"https://github.com/elastic/kibana/pull/191116","number":191116,"mergeCommit":{"message":"[Security Solution] Test plans for prebuilt rule import and export (#191116)\n\n## Summary\r\n\r\nThis PR introduces test plans for both [Prebuilt Rule\r\nImport](#180168) (corresponding\r\n[PR](#190198)) and [Prebuilt Rule\r\nExport](#180167) (corresponding\r\n[PR](#194498)). Import is\r\nconsiderably more complicated as it is calculating new values (for\r\n`rule_source`, `immutable`), while the export work is mainly removing\r\nexisting restrictions (which allowed only custom rules to be exported).\r\n\r\n---------\r\n\r\nCo-authored-by: Elastic Machine <[email protected]>","sha":"e4298492b5e48338396618d51168ea3e8427c103"}},{"branch":"8.x","label":"v8.17.0","branchLabelMappingKey":"^v8.17.0$","isSourceBranch":false,"state":"NOT_CREATED"}]}] BACKPORT--> Co-authored-by: Ryland Herrick <[email protected]>
tkajtoch
pushed a commit
to tkajtoch/kibana
that referenced
this pull request
Nov 12, 2024
…lastic#191116) ## Summary This PR introduces test plans for both [Prebuilt Rule Import](elastic#180168) (corresponding [PR](elastic#190198)) and [Prebuilt Rule Export](elastic#180167) (corresponding [PR](elastic#194498)). Import is considerably more complicated as it is calculating new values (for `rule_source`, `immutable`), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported). --------- Co-authored-by: Elastic Machine <[email protected]>
This was referenced Nov 14, 2024
CAWilson94
pushed a commit
to CAWilson94/kibana
that referenced
this pull request
Nov 18, 2024
…lastic#191116) ## Summary This PR introduces test plans for both [Prebuilt Rule Import](elastic#180168) (corresponding [PR](elastic#190198)) and [Prebuilt Rule Export](elastic#180167) (corresponding [PR](elastic#194498)). Import is considerably more complicated as it is calculating new values (for `rule_source`, `immutable`), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported). --------- Co-authored-by: Elastic Machine <[email protected]>
5 tasks
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
This PR introduces test plans for both Prebuilt Rule Import (corresponding PR) and Prebuilt Rule Export (corresponding PR). Import is considerably more complicated as it is calculating new values (for
rule_source,immutable), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported).