Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Threshold rule alert suppression [Classic] #4467

Merged
merged 6 commits into from
Jan 3, 2024
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
29 changes: 19 additions & 10 deletions docs/detections/alert-suppression.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -9,33 +9,42 @@ Alert suppression requires a https://www.elastic.co/pricing[Platinum or higher s
preview::[]
--

Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by a <<create-custom-rule,custom query>> detection rule.
Alert suppression allows you to reduce the number of repeated or duplicate detection alerts created by these detection rule types:

Normally, when a rule matches multiple source events, it creates multiple alerts, one for each event. When alert suppression is configured, matching events are grouped by their values in a specified field, and only one alert is created for each group. You can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values.
* <<create-custom-rule,Custom query>>
* <<create-threshold-rule,Threshold>>

Normally, when a rule meets its criteria repeatedly, it creates multiple alerts, one for each time the rule's criteria are met. When alert suppression is configured, duplicate qualifying events are grouped, and only one alert is created for each group. Depending on the rule type, you can configure alert suppression to create alerts each time the rule runs, or once within a specified time window. You can also specify multiple fields to group events by unique combinations of values.

The {security-app} displays several indicators in the Alerts table and the alert details flyout when a detection alert is created with alert suppression enabled. You can view the original events associated with suppressed alerts by investigating the alert in Timeline.

NOTE: Alert suppression is not available for Elastic prebuilt rules. However, if you want to suppress alerts for a prebuilt rule, you can duplicate it, then configure alert suppression on the duplicated rule.

=== Configure alert suppression

You can configure alert suppression when you create or edit a custom query rule. Refer to <<create-custom-rule>> for detailed instructions.
You can configure alert suppression when you create or edit a supported rule type. Refer to <<create-custom-rule>> or <<create-threshold-rule>> for detailed instructions.

. When configuring the rule type (the *Define rule* step for a new rule, or the *Definition* tab for an existing rule), enter one or more field names in *Suppress alerts by*.
. When configuring the rule type (the *Define rule* step for a new rule, or the *Definition* tab for an existing rule), specify how you want to group events for alert suppression:
+
NOTE: If you specify a field with multiple values, an alert grouping is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each matching value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`.
. Select how often to create alerts for duplicate events:
--
* Custom query rule: In *Suppress alerts by*, enter 1-3 field names to group events by the fields' values.
* Threshold rule: In *Group by*, enter up to 3 field names to group events by the fields' values, or leave the setting empty to group all qualifying events together.
--
+
NOTE: If you specify a field with multiple values, an alert grouping is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`.

. If available, select how often to create alerts for duplicate events:
+
--
* *Per rule execution*: Create an alert each time the rule runs and matches one or more events.
* *Per time period*: Create one alert for all matching events within a specified time window, beginning when the rule first matches an event and creates the alert.
* *Per rule execution*: (Custom query rules only) Create an alert each time the rule runs and meets its criteria.
* *Per time period*: Create one alert for all qualifying events within a specified time window, beginning when the rule first meets its criteria and creates the alert. (This is the only option available for threshold rules.)
+
For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule matches an event, it creates an alert at that time, and for the next hour, it'll suppress any subsequent matches.
For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule meets its criteria, it creates an alert at that time, and for the next hour, it'll suppress any subsequent qualifying events.
+
image::images/alert-suppression-options.png[Alert suppression options,400]
--

. Under *If a suppression field is missing*, choose how to handle events with missing suppression fields (events in which one or more of the *Suppress alerts by* fields don't exist):
. (Custom query rules only) Under *If a suppression field is missing*, choose how to handle events with missing suppression fields (events in which one or more of the *Suppress alerts by* fields don't exist):
* *Suppress and group alerts for events with missing fields*: Create one alert for each group of events with missing fields. Missing fields get a `null` value, which is used to group and suppress alerts.
* *Do not suppress alerts for events with missing fields*: Create a separate alert for each matching event. This basically falls back to normal alert creation for events with missing suppression fields.
vitaliidm marked this conversation as resolved.
Show resolved Hide resolved

Expand Down
4 changes: 2 additions & 2 deletions docs/detections/alerts-reduce.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -19,9 +19,9 @@ Use to avoid false alarms and unnecessary notifications during planned outages.
NOTE: Maintenance windows are a {kib} feature, configured outside of the {security-app} in *Stack Management*.

| <<alert-suppression,Alert suppression>>
a| *_Reduces repeated or duplicate alerts created by a custom query rule_*.
a| *_Reduces repeated or duplicate alerts_*.

Use to reduce the number of alerts created by a <<create-custom-rule,custom query rule>> that matches multiple source events. Matching events are grouped by their values in a specified field, and only one alert is created for each group.
Use to reduce the number of alerts created when a rule meets its criteria repeatedly. Duplicate qualifying events are grouped, and only one alert is created for each group.

| <<detections-ui-exceptions,Rule exception>>
a| *_Prevents a rule from creating alerts under specific conditions_*.
Expand Down
38 changes: 22 additions & 16 deletions docs/detections/rules-ui-create.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -81,22 +81,18 @@ When you use a saved query, the *Load saved query "_query name_" dynamically on

* Deselect this to load the saved query as a one-time way of populating the rule's *Custom query* field and filters. This copies the settings from the saved query to the rule, so you can then further adjust the rule's query and filters as needed. If the saved query is later changed, the rule will not inherit those changes.

. preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule.
. preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Use *Suppress alerts by* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <<alert-suppression>> for more information.

.. Enter a field name to group matching source events by the field's unique values; only one alert will be created for each group of events. You can also enter multiple fields to group events by unique combinations of values.
.. Enter a field name to group qualifying source events by the field's unique values; only one alert will be created for each group of events. You can also enter up to 3 fields to group events by unique combinations of values.
+
NOTE: If you specify a field with multiple values, an alert grouping is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each matching value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`.
NOTE: If you specify a field with multiple values, an alert grouping is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`.

.. Select how often to create alerts for duplicate events:

* *Per rule execution*: Create an alert each time the rule runs and meets its criteria.
* *Per time period*: Create one alert for all qualifying events within a specified time window, beginning when the rule first meets its criteria and creates the alert.
+
--
* *Per rule execution*: Create an alert each time the rule runs and matches duplicate events.
* *Per time period*: Create one alert for all matching events within a specified time window, beginning when the rule first matches an event and creates the alert.
+
For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule matches an event, it creates an alert at that time, and for the next hour any subsequent matches will be suppressed.
--
+
With either option, you can view all events associated with suppressed alerts by investigating the alert in Timeline. Refer to <<alert-suppression>> for more information.
For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule meets its criteria, it creates an alert at that time, and for the next hour, it'll suppress any subsequent qualifying events.

. Click **Continue** to <<rule-ui-basic-params, configure basic rule settings>>.

Expand All @@ -111,14 +107,24 @@ alerts.
+
NOTE: You can use {kib} saved queries (image:images/saved-query-menu.png[Saved query menu,18,18]) and queries from saved Timelines (*Import query from saved Timeline*) as rule conditions.

.. Use the `Group by` and `Threshold` fields to determine which source event field is used as a threshold and the threshold's value.
.. Use the `Count` field to limit alerts by cardinality of a certain field.
.. Use the *Group by* and *Threshold* fields to determine which source event field is used as a threshold and the threshold's value.
.. Use the *Count* field to limit alerts by cardinality of a certain field.
+
For example, if `Group by` is `source.ip`, `destination.ip` and its `Threshold` is `10`, an alert is generated for every pair of source and destination IP addresses that appear in at least 10 of the rule's search results.
For example, if *Group by* is `source.ip, destination.ip` and its *Threshold* is `10`, an alert is generated for every pair of source and destination IP addresses that appear in at least 10 of the rule's search results.
+
You can also leave the `Group by` field undefined. The rule then creates an alert when the number of search results is equal to or greater than the threshold value. If you set `Count` to limit the results by `process.name` >= 2, an alert will only be generated for source/destination IP pairs that appear with at least 2 unique process names across all events.
You can also leave the *Group by* field undefined. The rule then creates an alert when the number of search results is equal to or greater than the threshold value. If you set *Count* to limit the results by `process.name` >= 2, an alert will only be generated for source/destination IP pairs that appear with at least 2 unique process names across all events.
+
IMPORTANT: Alerts created by threshold rules are synthetic alerts that do not resemble the source documents. The alert itself only contains data about the fields that were aggregated over (the *Group by* fields). Other fields are omitted, because they can vary across all source documents that were counted toward the threshold. Additionally, you can reference the actual count of documents that exceeded the threshold from the `kibana.alert.threshold_result.count` field.

. preview:[] (Optional, https://www.elastic.co/pricing[Platinum or higher subscription] required) Select *Suppress alerts* to reduce the number of repeated or duplicate alerts created by the rule. Refer to <<alert-suppression>> for more information.

.. Enter field names in *Group by* to group qualifying source events by the fields' unique values; only one alert will be created for each group of events. You can enter up to 3 fields to group events by unique combinations of values. You can also leave *Group by* empty to group all qualifying events together.
+
NOTE: If you specify a field with multiple values, an alert grouping is created for each value. For example, if you suppress alerts by `destination.ip` of `[127.0.0.1, 127.0.0.2, 127.0.0.3]`, alerts will be suppressed separately for each value of `127.0.0.1`, `127.0.0.2`, and `127.0.0.3`.

.. In *Per time period*, specify how often to create alerts for duplicate events. This will create one alert for all qualifying events within the specified time window, beginning when the rule first meets its criteria and creates the alert.
+
IMPORTANT: Alerts created by threshold rules are synthetic alerts that do not resemble the source documents. The alert itself only contains data about the fields that were aggregated over (the `Group by` fields). Other fields are omitted, because they can vary across all source documents that were counted toward the threshold. Additionally, you can reference the actual count of documents that exceeded the threshold from the `kibana.alert.threshold_result.count` field.
For example, if a rule runs every 5 minutes but you don't need alerts that frequently, you can set the suppression time period to a longer time, such as 1 hour. If the rule meets its criteria, it creates an alert at that time, and for the next hour, it'll suppress any subsequent qualifying events.

. Click *Continue* to <<rule-ui-basic-params, configure basic rule settings>>.

Expand Down