forked from elastic/kibana
-
Notifications
You must be signed in to change notification settings - Fork 0
Commit
This commit does not belong to any branch on this repository, and may belong to a fork outside of the repository.
[Security Solution] Test plans for prebuilt rule import and export (e…
…lastic#191116) ## Summary This PR introduces test plans for both [Prebuilt Rule Import](elastic#180168) (corresponding [PR](elastic#190198)) and [Prebuilt Rule Export](elastic#180167) (corresponding [PR](elastic#194498)). Import is considerably more complicated as it is calculating new values (for `rule_source`, `immutable`), while the export work is mainly removing existing restrictions (which allowed only custom rules to be exported). --------- Co-authored-by: Elastic Machine <[email protected]>
- Loading branch information
1 parent
ef2ad21
commit 0057631
Showing
2 changed files
with
192 additions
and
0 deletions.
There are no files selected for viewing
65 changes: 65 additions & 0 deletions
65
...solution/docs/testing/test_plans/detection_response/prebuilt_rules/exporting.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,65 @@ | ||
| # Prebuilt Rule Export | ||
|
|
||
| This is a test plan for the exporting of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic. | ||
|
|
||
| Status: `in progress`. | ||
|
|
||
| ## Useful information | ||
|
|
||
| ### Tickets | ||
|
|
||
| - [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) | ||
| - [Rule Exporting Feature](https://github.com/elastic/kibana/issues/180167#issue-2227974379) | ||
| - [Rule Export API PR](https://github.com/elastic/kibana/pull/194498) | ||
|
|
||
| ### Terminology | ||
|
|
||
| - **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet. | ||
| - **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules | ||
| - **rule source, or ruleSource**: A field on the rule that defines the rule's categorization | ||
|
|
||
| ## Scenarios | ||
|
|
||
| ### Core Functionality | ||
|
|
||
| #### Scenario: Exporting prebuilt rule individually | ||
| ```Gherkin | ||
| Given a space with prebuilt rules installed | ||
| When the user selects "Export rule" from the "All actions" dropdown on the rule's page | ||
| Then the rule should be exported as an NDJSON file | ||
| And it should include an "immutable" field with a value of true | ||
| And its "ruleSource" "type" should be "external" | ||
| And its "ruleSource" "isCustomized" value should depend on whether the rule was customized | ||
| ``` | ||
|
|
||
| #### Scenario: Exporting prebuilt rules in bulk | ||
| ```Gherkin | ||
| Given a space with prebuilt rules installed | ||
| When the user selects prebuilt rules in the alerts table | ||
| And chooses "Export" from bulk actions | ||
| Then the selected rules should be exported as an NDJSON file | ||
| And they should include an "immutable" field with a value of true | ||
| And their "ruleSource" "type" should be "external" | ||
| And their "ruleSource" "isCustomized" should depend on whether the rule was customized | ||
| ``` | ||
|
|
||
| #### Scenario: Exporting both prebuilt and custom rules in bulk | ||
| ```Gherkin | ||
| Given a space with prebuilt and custom rules installed | ||
| When the user selects prebuilt rules in the alerts table | ||
| And chooses "Export" from bulk actions | ||
| Then the selected rules should be exported as an NDJSON file | ||
| And the prebuilt rules should include an "immutable" field with a value of true | ||
| And the custom rules should include an "immutable" field with a value of false | ||
| And the prebuilt rules' "ruleSource" "type" should be "external" | ||
| And the custom rules' "ruleSource" "type" should be "internal" | ||
| ``` | ||
|
|
||
| ### Error Handling | ||
|
|
||
| #### Scenario: Exporting beyond the export limit | ||
| ```Gherkin | ||
| Given a space with prebuilt and custom rules installed | ||
| And the number of rules is greater than the export limit (defaults to 10_000) | ||
| Then the request should be rejected as a bad request | ||
| ``` |
127 changes: 127 additions & 0 deletions
127
...solution/docs/testing/test_plans/detection_response/prebuilt_rules/importing.md
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,127 @@ | ||
| # Prebuilt Rule Import | ||
|
|
||
| This is a test plan for the importing of prebuilt rules. This feature is an aspect of `Milestone 2` of the [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) epic. | ||
|
|
||
| Status: `in progress`. | ||
|
|
||
| ## Useful information | ||
|
|
||
| ### Tickets | ||
|
|
||
| - [Rule Immutability/Customization](https://github.com/elastic/security-team/issues/1974) | ||
| - [Rule Importing Feature](https://github.com/elastic/kibana/issues/180168) | ||
| - [Rule Import API PR](https://github.com/elastic/kibana/pull/190198) | ||
|
|
||
| ### Terminology | ||
|
|
||
| - **prebuilt rule**: A rule contained in our `Prebuilt Security Detection Rules` integration in Fleet. | ||
| - **custom rule**: A rule defined by the user, which has no relation to the prebuilt rules | ||
| - **rule source, or ruleSource**: A field on the rule that defines the rule's categorization | ||
|
|
||
| ## Scenarios | ||
|
|
||
| ### Core Functionality | ||
|
|
||
| #### Scenario: Importing an unmodified prebuilt rule with a matching rule_id and version | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule with a matching rule_id and version, identical to the published rule | ||
| When the user imports the rule | ||
| Then the rule should be created or updated | ||
| And the ruleSource type should be "external" | ||
| And isCustomized should be false | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a customized prebuilt rule with a matching rule_id and version | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule with a matching rule_id and version, modified from the published version | ||
| When the user imports the rule | ||
| Then the rule should be created or updated | ||
| And the ruleSource type should be "external" | ||
| And isCustomized should be true | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a prebuilt rule with a matching rule_id but no matching version | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule with a matching rule_id but no matching version | ||
| When the user imports the rule | ||
| Then the rule should be created or updated | ||
| And the ruleSource type should be "external" | ||
| And isCustomized should be true | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a prebuilt rule with a non-existent rule_id | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule with a non-existent rule_id | ||
| When the user imports the rule | ||
| Then the rule should be created | ||
| And the ruleSource type should be "internal" | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a prebuilt rule without a rule_id field | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule without a rule_id field | ||
| When the user imports the rule | ||
| Then the import should be rejected with a message "rule_id field is required" | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a prebuilt rule with a matching rule_id but missing a version field | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a prebuilt rule without a version field | ||
| When the user imports the rule | ||
| Then the import should be rejected with a message "version field is required" | ||
| ``` | ||
|
|
||
| #### Scenario: Importing an existing custom rule missing a version field | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains an existing custom rule without a version field | ||
| When the user imports the rule | ||
| Then the rule should be updated | ||
| And the ruleSource type should be "internal" | ||
| And the "version" field should be set to the existing rule's "version" | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a new custom rule missing a version field | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a new custom rule without a version field | ||
| When the user imports the rule | ||
| Then the rule should be created | ||
| And the ruleSource type should be "internal" | ||
| And the "version" field should be set to 1 | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a rule with overwrite flag set to true | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a rule with an existing rule_id | ||
| And the overwrite flag is set to true | ||
| When the user imports the rule | ||
| Then the rule should be overwritten | ||
| And the ruleSource type should be calculated based on the rule_id and version | ||
| ``` | ||
|
|
||
| #### Scenario: Importing a rule with overwrite flag set to false | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains a rule with an existing rule_id | ||
| And the overwrite flag is set to false | ||
| When the user imports the rule | ||
| Then the import should be rejected with a message "rule_id already exists" | ||
| ``` | ||
|
|
||
| #### Scenario: Importing both custom and prebuilt rules | ||
|
|
||
| ```Gherkin | ||
| Given the import payload contains modified and unmodified, custom and prebuilt rules | ||
| When the user imports the rule | ||
| Then custom rules should be created or updated, with versions defaulted to 1 | ||
| And prebuilt rules should be created or updated, | ||
| And prebuilt rules missing versions should be rejected | ||
| ``` |