From 943690e1462ac2dca1a8956c0ea89539bd859ea4 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 16 Jul 2024 20:09:17 -0400 Subject: [PATCH 01/11] First draft --- docs/detections/alert-suppression.asciidoc | 11 +++++------ docs/detections/api/rules/rules-api-create.asciidoc | 6 +++--- docs/detections/api/rules/rules-api-update.asciidoc | 6 +++--- docs/serverless/alerts/alert-suppression.mdx | 11 +++++------ 4 files changed, 16 insertions(+), 18 deletions(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index d620f7bd4e..8a43037bd4 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -16,6 +16,8 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * <> * <> (non-sequence queries only) * <> +* <> +* <> Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. @@ -25,16 +27,13 @@ NOTE: Alert suppression is not available for Elastic prebuilt rules. However, if === Configure alert suppression -You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating <>, <>, <>, or <> rules for detailed instructions. +You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating <>, <>, <>, <>, <>, or <> rules for detailed instructions. . When configuring the rule type (the *Define rule* step for a new rule, or the *Definition* tab for an existing rule), specify how you want to group events for alert suppression: + -- -* Custom query rule: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. +* Custom query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. * Threshold rule: In *Group by*, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. -* Indicator match rule: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. -* Event correlation rule (non-sequence queries only): In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. -* New terms rule: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. -- + @@ -113,5 +112,5 @@ image::images/timeline-button.png[Investigate in timeline button, 200] Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): -* **Threshold and event correlation (non-sequence queries only)** - The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. +* **Threshold, event correlation (non-sequence queries only), {esql}, and {ml}** - The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. * **Indicator match and new terms** - The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new term rules is `500`. \ No newline at end of file diff --git a/docs/detections/api/rules/rules-api-create.asciidoc b/docs/detections/api/rules/rules-api-create.asciidoc index 82ea9e8ae5..a3177f645f 100644 --- a/docs/detections/api/rules/rules-api-create.asciidoc +++ b/docs/detections/api/rules/rules-api-create.asciidoc @@ -503,11 +503,11 @@ a detection rule exception (`detection`) or an endpoint exception (`endpoint`). |============================================== [[opt-fields-alert-suppression-create]] -===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), and new terms rules +===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules -preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, and new terms rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] +preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, new terms, {ml}, and {esql} rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] -====== Query, indicator match, event correlation (non-sequence queries only), and new terms rules +====== Query, indicator match, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules [width="100%",options="header"] |============================================== diff --git a/docs/detections/api/rules/rules-api-update.asciidoc b/docs/detections/api/rules/rules-api-update.asciidoc index 3ce24d139c..dd977cd4c0 100644 --- a/docs/detections/api/rules/rules-api-update.asciidoc +++ b/docs/detections/api/rules/rules-api-update.asciidoc @@ -532,11 +532,11 @@ in the UI (*Rules* -> *Detection rules (SIEM)* -> *_Rule name_*). [[opt-fields-alert-suppression-update]] -===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), and new terms rules +===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules -preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, and new terms rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] +preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, new terms, {ml}, and {esql} rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] -====== Query, indicator match, event correlation (non-sequence queries only), and new terms rules +====== Query, indicator match, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules [width="100%",options="header"] |============================================== diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index 1a9d1dd6c3..82a30bbb21 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -20,6 +20,8 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * * (non-sequence queries only) * +* +* Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. @@ -31,15 +33,12 @@ Alert suppression is not available for Elastic prebuilt rules. However, if you w ## Configure alert suppression -You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating , , , , or rules for detailed instructions. +You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating , , , , , , or rules for detailed instructions. 1. When configuring the rule type (the **Define rule** step for a new rule, or the **Definition** tab for an existing rule), specify how you want to group events for alert suppression: - * Custom query rule: In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. + * Custom query rule, indicator match, threshold, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rules: In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. * Threshold rule: In **Group by**, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. - * Indicator match rule: In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. - * Event correlation rule (non-sequence queries only): In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. - * New terms rule: In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. If you specify a field with multiple values, alerts with that field are handled as follows: @@ -112,5 +111,5 @@ With alert suppression, detection alerts aren't created for the grouped source e Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): -* **Threshold and event correlation (non-sequence queries only)** - The maximum number is the value you choose for the rule's **Max alerts per run** advanced setting, which is `100` by default. +* **Threshold, event correlation (non-sequence queries only, ((esql)), and ((ml))** - The maximum number is the value you choose for the rule's **Max alerts per run** advanced setting, which is `100` by default. * **Indicator match and new terms** - The maximum number is five times the value you choose for the the rule's **Max alerts per run** advanced setting. The default value is `100`, which means the default maximum limit for indicator match rules and new terms rules is `500`. \ No newline at end of file From 4eac0accbf636f3222b832b91d1750d0211db0d2 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 16 Jul 2024 22:19:31 -0400 Subject: [PATCH 02/11] More changes --- docs/detections/alert-suppression.asciidoc | 10 +++++----- docs/detections/api/rules/rules-api-create.asciidoc | 4 ++-- docs/detections/api/rules/rules-api-update.asciidoc | 4 ++-- docs/detections/rules-ui-create.asciidoc | 7 +++++++ docs/serverless/alerts/alert-suppression.mdx | 4 ++-- 5 files changed, 18 insertions(+), 11 deletions(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index 8a43037bd4..f6105e8393 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -17,7 +17,7 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * <> (non-sequence queries only) * <> * <> -* <> +* <> Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. @@ -32,8 +32,8 @@ You can configure alert suppression when you create or edit a supported rule typ . When configuring the rule type (the *Define rule* step for a new rule, or the *Definition* tab for an existing rule), specify how you want to group events for alert suppression: + -- -* Custom query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. -* Threshold rule: In *Group by*, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. +* **Custom query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules:** In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values. +* **Threshold rule:** In *Group by*, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. -- + @@ -41,8 +41,8 @@ You can configure alert suppression when you create or edit a supported rule typ ====== If you specify a field with multiple values, alerts with that field are handled as follows: -* **Custom query or threshold rules** - A group of alerts is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. -* **Indicator match rule, event correlation (non-sequence queries only), or new terms rule** - Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. +* **Custom query or threshold rules:** A group of alerts is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. +* **Indicator match rule, event correlation (non-sequence queries only), new terms, or {esql} rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. ====== diff --git a/docs/detections/api/rules/rules-api-create.asciidoc b/docs/detections/api/rules/rules-api-create.asciidoc index a3177f645f..a39dfefb66 100644 --- a/docs/detections/api/rules/rules-api-create.asciidoc +++ b/docs/detections/api/rules/rules-api-create.asciidoc @@ -503,11 +503,11 @@ a detection rule exception (`detection`) or an endpoint exception (`endpoint`). |============================================== [[opt-fields-alert-suppression-create]] -===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules +===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {esql}, and {ml} rules preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, new terms, {ml}, and {esql} rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] -====== Query, indicator match, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules +====== Query, indicator match, event correlation (non-sequence queries only), new terms, {esql}, and {ml} rules [width="100%",options="header"] |============================================== diff --git a/docs/detections/api/rules/rules-api-update.asciidoc b/docs/detections/api/rules/rules-api-update.asciidoc index dd977cd4c0..cac28ad2ff 100644 --- a/docs/detections/api/rules/rules-api-update.asciidoc +++ b/docs/detections/api/rules/rules-api-update.asciidoc @@ -532,11 +532,11 @@ in the UI (*Rules* -> *Detection rules (SIEM)* -> *_Rule name_*). [[opt-fields-alert-suppression-update]] -===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules +===== Optional alert suppression fields for query, indicator match, threshold, event correlation (non-sequence queries only), new terms, {esql}, and {ml} rules preview::["Alert suppression is in technical preview for threshold, indicator match, event correlation, new terms, {ml}, and {esql} rules. The functionality may be changed or removed in a future release. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features."] -====== Query, indicator match, event correlation (non-sequence queries only), new terms, {ml}, and {esql} rules +====== Query, indicator match, event correlation (non-sequence queries only), new terms, {esql}, and {ml} rules [width="100%",options="header"] |============================================== diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index d9274bf131..8e300763cc 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -46,6 +46,8 @@ then select: NOTE: If a required job isn't currently running, it will automatically start when you finish configuring and enable the rule. .. The anomaly score threshold above which alerts are created. + +. preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. ++ //// The following step is repeated across all rule types. If you change anything @@ -353,6 +355,9 @@ NOTE: Refer to the sections below to learn more about <> for more information. ++ + //// The following steps are repeated across multiple rule types. If you change anything in these steps or sub-steps, apply the change to the other rule types, too. @@ -466,6 +471,8 @@ If the `LIMIT` value and **Max alerts per run** value are different, the rule us - When writing an aggregating query, use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. +- When configuring alert suppression on a non-aggregating query, avoid sorting alerts by descending `@timestamp` order. Doing so can impact the number of alerts that are suppressed, especially if the number of alerts generated is higher than the maximum number of alerts the rule can create when it runs. + [float] [[esql-rule-limitations]] ==== {esql} rule limitations diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index 82a30bbb21..2a044acca8 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -43,8 +43,8 @@ You can configure alert suppression when you create or edit a supported rule typ If you specify a field with multiple values, alerts with that field are handled as follows: - * **Custom query or threshold rules** - Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. - * **Indicator match, event correlation (non-sequence queries only), or new terms rule** - Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. + * **Custom query or threshold rules:** Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. + * **Indicator match, event correlation (non-sequence queries only), new terms, or ((esql)) rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. From 6dcbd05a5d492b75bcd5b98f1cb7f5dc9a85f756 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 16 Jul 2024 22:37:35 -0400 Subject: [PATCH 03/11] Fixing styles --- docs/detections/alert-suppression.asciidoc | 4 ++-- docs/serverless/alerts/alert-suppression.mdx | 10 +++++----- docs/serverless/rules/rules-ui-create.mdx | 6 ++++++ 3 files changed, 13 insertions(+), 7 deletions(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index f6105e8393..ec8fc8b4ae 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -112,5 +112,5 @@ image::images/timeline-button.png[Investigate in timeline button, 200] Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): -* **Threshold, event correlation (non-sequence queries only), {esql}, and {ml}** - The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. -* **Indicator match and new terms** - The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new term rules is `500`. \ No newline at end of file +* **Threshold, event correlation (non-sequence queries only), {esql}, and {ml}:** The maximum number of alerts is the value you choose for the rule's **Max alerts per run** <>, which is `100` by default. +* **Indicator match and new terms:** The maximum number is five times the value you choose for the rule's **Max alerts per run** <>. The default value is `100`, which means the default maximum limit for indicator match rules and new term rules is `500`. \ No newline at end of file diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index 2a044acca8..e0159a946d 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -20,8 +20,8 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * * (non-sequence queries only) * -* -* +* +* Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. @@ -33,7 +33,7 @@ Alert suppression is not available for Elastic prebuilt rules. However, if you w ## Configure alert suppression -You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating , , , , , , or rules for detailed instructions. +You can configure alert suppression when you create or edit a supported rule type. Refer to documentation for creating , , , , , , or rules for detailed instructions. 1. When configuring the rule type (the **Define rule** step for a new rule, or the **Definition** tab for an existing rule), specify how you want to group events for alert suppression: @@ -111,5 +111,5 @@ With alert suppression, detection alerts aren't created for the grouped source e Some rule types have a maximum number of alerts that can be suppressed (custom query rules don't have a suppression limit): -* **Threshold, event correlation (non-sequence queries only, ((esql)), and ((ml))** - The maximum number is the value you choose for the rule's **Max alerts per run** advanced setting, which is `100` by default. -* **Indicator match and new terms** - The maximum number is five times the value you choose for the the rule's **Max alerts per run** advanced setting. The default value is `100`, which means the default maximum limit for indicator match rules and new terms rules is `500`. \ No newline at end of file +* **Threshold, event correlation (non-sequence queries only, ((esql)), and ((ml)):** The maximum number is the value you choose for the rule's **Max alerts per run** advanced setting, which is `100` by default. +* **Indicator match and new terms:** The maximum number is five times the value you choose for the the rule's **Max alerts per run** advanced setting. The default value is `100`, which means the default maximum limit for indicator match rules and new terms rules is `500`. \ No newline at end of file diff --git a/docs/serverless/rules/rules-ui-create.mdx b/docs/serverless/rules/rules-ui-create.mdx index 984d0c3cb4..5535747062 100644 --- a/docs/serverless/rules/rules-ui-create.mdx +++ b/docs/serverless/rules/rules-ui-create.mdx @@ -49,6 +49,8 @@ To create or edit ((ml)) rules, you need an appropriate user role. Additionally, 1. The anomaly score threshold above which alerts are created. +1. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to Suppress detection alerts for more information. + {/* The following steps are repeated across multiple rule types. If you change anything in these steps or sub-steps, apply the change to the other rule types, too. */} 1. (Optional) Add **Related integrations** to associate the rule with one or more [Elastic integrations](((integrations-docs))). This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's installation status when viewing the rule. @@ -383,6 +385,8 @@ To create an ((esql)) rule: Click the help icon () to open the in-product reference documentation for all ((esql)) commands and functions. +1. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to Suppress detection alerts for more information. + {/* The following steps are repeated across multiple rule types. If you change anything in these steps or sub-steps, apply the change to the other rule types, too. */} 1. (Optional) Create a list of **Required fields** that the rule needs to function. This list is informational only, to help users understand the rule; it doesn't affect how the rule actually runs. @@ -500,6 +504,8 @@ When writing your query, consider the following: - When writing an aggregating query, use the [`STATS...BY`](((ref))/esql-commands.html#esql-stats-by) command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. +- When configuring alert suppression on a non-aggregating query, avoid sorting alerts by descending `@timestamp` order. Doing so can impact the number of alerts that are suppressed, especially if the number of alerts generated is higher than the maximum number of alerts the rule can create when it runs. +
### ((esql)) rule limitations From ba9d59780bed50053b3c70cb76f9877496085987 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 16 Jul 2024 23:01:44 -0400 Subject: [PATCH 04/11] Fixed attribute --- docs/detections/alert-suppression.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index ec8fc8b4ae..de0ba0b820 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -17,7 +17,7 @@ Alert suppression allows you to reduce the number of repeated or duplicate detec * <> (non-sequence queries only) * <> * <> -* <> +* <> Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values. From bb9314c94b445bc7266970f1ad8b6e60ef514527 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Tue, 16 Jul 2024 23:12:20 -0400 Subject: [PATCH 05/11] Style update --- docs/serverless/alerts/alert-suppression.mdx | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index e0159a946d..159ba52d66 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -37,8 +37,8 @@ You can configure alert suppression when you create or edit a supported rule typ 1. When configuring the rule type (the **Define rule** step for a new rule, or the **Definition** tab for an existing rule), specify how you want to group events for alert suppression: - * Custom query rule, indicator match, threshold, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rules: In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. - * Threshold rule: In **Group by**, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. + * **Custom query rule, indicator match, threshold, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rules:** In **Suppress alerts by**, enter 1-3 field names to group events by the fields' values. + * **Threshold rule:** In **Group by**, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together. If you specify a field with multiple values, alerts with that field are handled as follows: From 893ab42f59ed03a3488dc16698d940a137cdfda0 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Wed, 17 Jul 2024 09:30:23 -0400 Subject: [PATCH 06/11] Vitalii's input --- docs/detections/alert-suppression.asciidoc | 4 ++-- docs/detections/rules-ui-create.asciidoc | 2 +- docs/serverless/alerts/alert-suppression.mdx | 4 ++-- docs/serverless/rules/rules-ui-create.mdx | 2 +- 4 files changed, 6 insertions(+), 6 deletions(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index de0ba0b820..c49cb28ee0 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -42,13 +42,13 @@ You can configure alert suppression when you create or edit a supported rule typ If you specify a field with multiple values, alerts with that field are handled as follows: * **Custom query or threshold rules:** A group of alerts is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. -* **Indicator match rule, event correlation (non-sequence queries only), new terms, or {esql} rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. +* **Indicator match rule, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. ====== . If available, select how often to create alerts for duplicate events: + -NOTE: Both options are available for custom query, indicator match, event correlation, and new terms rules. Threshold rules only have the *Per time period* option. +NOTE: Both options are available for custom query, indicator match, event correlation, new terms, {esql}, and {ml} rules. Threshold rules only have the *Per time period* option. + -- * *Per rule execution*: Create an alert each time the rule runs and meets its criteria. diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 8e300763cc..fff8475118 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -471,7 +471,7 @@ If the `LIMIT` value and **Max alerts per run** value are different, the rule us - When writing an aggregating query, use the {ref}/esql-commands.html#esql-stats-by[`STATS...BY`] command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. -- When configuring alert suppression on a non-aggregating query, avoid sorting alerts by descending `@timestamp` order. Doing so can impact the number of alerts that are suppressed, especially if the number of alerts generated is higher than the maximum number of alerts the rule can create when it runs. +- When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value. [float] [[esql-rule-limitations]] diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index 159ba52d66..4a49d30118 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -44,14 +44,14 @@ You can configure alert suppression when you create or edit a supported rule typ If you specify a field with multiple values, alerts with that field are handled as follows: * **Custom query or threshold rules:** Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. - * **Indicator match, event correlation (non-sequence queries only), new terms, or ((esql)) rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. + * **Indicator match, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. 1. If available, select how often to create alerts for duplicate events: - Both options are available for custom query, indicator match, event correlation, and new terms rules. Threshold rules only have the **Per time period** option. + Both options are available for custom query, indicator match, event correlation, new terms, ((esql)), and ((ml)) rules. Threshold rules only have the **Per time period** option. * **Per rule execution**: Create an alert each time the rule runs and an event meets its criteria. diff --git a/docs/serverless/rules/rules-ui-create.mdx b/docs/serverless/rules/rules-ui-create.mdx index 5535747062..4b4e8b154a 100644 --- a/docs/serverless/rules/rules-ui-create.mdx +++ b/docs/serverless/rules/rules-ui-create.mdx @@ -504,7 +504,7 @@ When writing your query, consider the following: - When writing an aggregating query, use the [`STATS...BY`](((ref))/esql-commands.html#esql-stats-by) command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. -- When configuring alert suppression on a non-aggregating query, avoid sorting alerts by descending `@timestamp` order. Doing so can impact the number of alerts that are suppressed, especially if the number of alerts generated is higher than the maximum number of alerts the rule can create when it runs. +-- When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value.
From c0dcb1ca13f0a113b7aad4d45977855a17e54dc7 Mon Sep 17 00:00:00 2001 From: "nastasha.solomon" Date: Thu, 18 Jul 2024 16:29:22 -0400 Subject: [PATCH 07/11] ML rule update --- docs/detections/rules-ui-create.asciidoc | 2 ++ docs/serverless/rules/rules-ui-create.mdx | 6 +++++- 2 files changed, 7 insertions(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index fff8475118..7ae83b9162 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -48,6 +48,8 @@ NOTE: If a required job isn't currently running, it will automatically start whe + . preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. + +NOTE: Because {ml} rules generate alerts from anomalies, alert documents don't contain source event fields and you can only use anomaly fields when configuring alert suppression. ++ //// The following step is repeated across all rule types. If you change anything diff --git a/docs/serverless/rules/rules-ui-create.mdx b/docs/serverless/rules/rules-ui-create.mdx index 4b4e8b154a..941c52c1cb 100644 --- a/docs/serverless/rules/rules-ui-create.mdx +++ b/docs/serverless/rules/rules-ui-create.mdx @@ -51,6 +51,10 @@ To create or edit ((ml)) rules, you need an appropriate user role. Additionally, 1. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to Suppress detection alerts for more information. + + Because ((ml)) rules generate alerts from anomalies, alert documents don't contain source event fields and you can only use anomaly fields when configuring alert suppression. + + {/* The following steps are repeated across multiple rule types. If you change anything in these steps or sub-steps, apply the change to the other rule types, too. */} 1. (Optional) Add **Related integrations** to associate the rule with one or more [Elastic integrations](((integrations-docs))). This indicates the rule's dependency on specific integrations and the data they generate, and allows users to confirm each integration's installation status when viewing the rule. @@ -504,7 +508,7 @@ When writing your query, consider the following: - When writing an aggregating query, use the [`STATS...BY`](((ref))/esql-commands.html#esql-stats-by) command with fields that you want to search and filter for after alerts are created. For example, using the `host.name`, `user.name`, `process.name` fields with the `BY` operator of the `STATS...BY` command returns these fields in alert documents, and allows you to search and filter for them from the Alerts table. --- When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value. +- When configuring alert suppression on a non-aggregating query, we recommend sorting results by ascending `@timestamp` order. Doing so ensures that alerts are properly suppressed, especially if the number of alerts generated is higher than the **Max alerts per run** value.
From 9d48e8b639856779c36ff17c5fd470d49bca7f35 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Thu, 18 Jul 2024 17:40:07 -0400 Subject: [PATCH 08/11] Update docs/detections/rules-ui-create.asciidoc Co-authored-by: Ryland Herrick --- docs/detections/rules-ui-create.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index 7ae83b9162..24aef2f7a6 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -48,7 +48,7 @@ NOTE: If a required job isn't currently running, it will automatically start whe + . preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <> for more information. + -NOTE: Because {ml} rules generate alerts from anomalies, alert documents don't contain source event fields and you can only use anomaly fields when configuring alert suppression. +NOTE: Because {ml} rules generate alerts from anomalies, which don't contain source event fields, you can only use anomaly fields when configuring alert suppression. + //// From 873f0aeda22fa27e8b0aba3dc0b4401934d6ff30 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Thu, 18 Jul 2024 17:40:57 -0400 Subject: [PATCH 09/11] Update docs/serverless/rules/rules-ui-create.mdx --- docs/serverless/rules/rules-ui-create.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/serverless/rules/rules-ui-create.mdx b/docs/serverless/rules/rules-ui-create.mdx index 941c52c1cb..bb3d8e84f1 100644 --- a/docs/serverless/rules/rules-ui-create.mdx +++ b/docs/serverless/rules/rules-ui-create.mdx @@ -52,7 +52,7 @@ To create or edit ((ml)) rules, you need an appropriate user role. Additionally, 1. (Optional) Use **Suppress alerts by** to reduce the number of repeated or duplicate alerts created by the rule. Refer to Suppress detection alerts for more information. - Because ((ml)) rules generate alerts from anomalies, alert documents don't contain source event fields and you can only use anomaly fields when configuring alert suppression. + Because ((ml)) rules generate alerts from anomalies, which don't contain source event fields, you can only use anomaly fields when configuring alert suppression. {/* The following steps are repeated across multiple rule types. If you change anything From e1f0d87603dcee8a45c5b0921e23c9f3730d50ed Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 22 Jul 2024 10:20:19 -0400 Subject: [PATCH 10/11] Update docs/detections/alert-suppression.asciidoc Co-authored-by: natasha-moore-elastic <137783811+natasha-moore-elastic@users.noreply.github.com> --- docs/detections/alert-suppression.asciidoc | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/detections/alert-suppression.asciidoc b/docs/detections/alert-suppression.asciidoc index c49cb28ee0..73f0537840 100644 --- a/docs/detections/alert-suppression.asciidoc +++ b/docs/detections/alert-suppression.asciidoc @@ -42,7 +42,7 @@ You can configure alert suppression when you create or edit a supported rule typ If you specify a field with multiple values, alerts with that field are handled as follows: * **Custom query or threshold rules:** A group of alerts is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. -* **Indicator match rule, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. +* **Indicator match, event correlation (non-sequence queries only), new terms, {esql}, or {ml} rules:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. ====== From 418c4052151507c5ca8c4b9515c2dda40d29dfe6 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Mon, 22 Jul 2024 10:20:51 -0400 Subject: [PATCH 11/11] Update docs/serverless/alerts/alert-suppression.mdx --- docs/serverless/alerts/alert-suppression.mdx | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/serverless/alerts/alert-suppression.mdx b/docs/serverless/alerts/alert-suppression.mdx index 4a49d30118..462c03402e 100644 --- a/docs/serverless/alerts/alert-suppression.mdx +++ b/docs/serverless/alerts/alert-suppression.mdx @@ -44,7 +44,7 @@ You can configure alert suppression when you create or edit a supported rule typ If you specify a field with multiple values, alerts with that field are handled as follows: * **Custom query or threshold rules:** Alerts are grouped by each unique value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`. - * **Indicator match, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rule:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group. + * **Indicator match, event correlation (non-sequence queries only), new terms, ((esql)), or ((ml)) rules:** Alerts with the specified field name and identical array values are grouped together. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts with the entire array are grouped and only one alert is created for the group.