Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[DOCS] Create rule updates 8.4 #2339

Merged
merged 7 commits into from
Aug 24, 2022
Merged
Show file tree
Hide file tree
Changes from 5 commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
Binary file modified docs/detections/images/create-new-rule.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
Binary file added docs/detections/images/saved-query-menu.png
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
51 changes: 38 additions & 13 deletions docs/detections/rules-ui-create.asciidoc
Original file line number Diff line number Diff line change
Expand Up @@ -29,13 +29,15 @@ alert is generated for every source IP address that appears in at least 10 of
the rule's search results.
* <<create-eql-rule, *Event correlation*>>: Searches the defined indices and creates an alert when results match an
{ref}/eql.html[Event Query Language (EQL)] query.
* <<create-indicator-rule, *Indicator match*>>: Creates an alert when {es-sec} index field values match field values defined in the specified indicator index patterns. For example, you can create an indicator index for IP addresses and use this index to create an alert whenever an event's `destination.ip` equals a value in the index. Indicator index field mappings should be {ecs-ref}[ECS-compliant]. For information on creating {es} indices and field types, see
* <<create-indicator-rule, *Indicator match*>>: Creates an alert when {elastic-sec} index field values match field values defined in the specified indicator index patterns. For example, you can create an indicator index for IP addresses and use this index to create an alert whenever an event's `destination.ip` equals a value in the index. Indicator index field mappings should be {ecs-ref}[ECS-compliant]. For information on creating {es} indices and field types, see
{ref}/getting-started-index.html[Index some documents],
{ref}/indices-create-index.html[Create index API], and
{ref}/mapping-types.html[Field data types]. If you have indicators in a standard file format, such as CSV or JSON, you can also use the Machine Learning Data Visualizer to import your indicators into an indicator index. See https://www.elastic.co/guide/en/machine-learning/7.10/ml-gs-visualizer.html and use the Import Data option to import your indicators.
{ref}/mapping-types.html[Field data types]. If you have indicators in a standard file format, such as CSV or JSON, you can also use the Machine Learning Data Visualizer to import your indicators into an indicator index. See {ml-docs}/ml-getting-started.html#sample-data-visualizer[Explore the data in {kib}] and use the *Import Data* option to import your indicators.

TIP: You can also use value lists as the indicator match index. See <<indicator-value-lists>> at the end of this topic for more information.

* <<create-new-terms-rule, *New terms*>>: Generates an alert for each new term it detects in source documents.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved

When modifying rules or managing detection alerts, you can add exceptions that prevent a rule from generating an alert even when its criteria are met. This is useful for reducing noise, such as preventing alerts from trusted processes and internal IP addresses. <<detections-ui-exceptions>> describes how to add exceptions to a rule.

NOTE: You can add exceptions to custom query, machine learning, event correlation, and indicator match rule types.
Expand Down Expand Up @@ -77,6 +79,14 @@ Rules, including all background detection and the actions they generate, are aut
If a rule requires certain privileges to run, such as index privileges, keep in mind that if a user without those privileges updates the rule, the rule will no longer function.
==============================================

[float]
[[views-index-patterns]]
=== Data views and index patterns

When you create a rule, you must specify the {es} index pattens for which you'd like the rule to run, or select a {kibana-ref}/data-views.html[data view] as the source of data. If you select a data view, when you create a rule (with the exception of a {ml} rule), you can select runtime fields associated with that data view.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved

NOTE: To access data views, ensure you have the {kibana-ref}/data-views.html#data-views-read-only-access[required permissions].

[float]
[[create-rule-ui]]
=== Select rule type and scope
Expand All @@ -93,6 +103,7 @@ image::images/create-new-rule.png[]
* <<create-threshold-rule, Threshold>>
* <<create-eql-rule, Event correlation>>
* <<create-indicator-rule, Indicator match>>
* <<create-new-terms-rule, New terms>>

[discrete]
[[create-ml-rule]]
Expand Down Expand Up @@ -123,11 +134,11 @@ image::images/rule-start-ml-job.png[]
==== Create a custom query rule
. To create a rule based on a KQL or Lucene query, select *Custom query*,
then:
.. Define which {es} indices the rule searches for alerts.
.. Define which {es} indices or data view the rule searches for alerts.
.. Use the filter and query fields to create the criteria used for detecting
alerts.
+
NOTE: You can use {kib} saved queries (save icon) and queries from saved Timelines (`Import query from saved Timeline`) as rule conditions.
NOTE: You can use {kib} saved queries (image:images/saved-query-menu.png[Saved query menu,17,17]) and queries from saved Timelines (`Import query from saved Timeline`) as rule conditions.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
+
For example, the following rule detects when the `vssadmin delete shadows`
Windows command is executed:
Expand Down Expand Up @@ -172,7 +183,7 @@ IMPORTANT: Alerts created by threshold rules are synthetic alerts that do not re
[[create-eql-rule]]
==== Create an event correlation rule
. To create an event correlation rule using EQL, select *Event Correlation*, then:
.. Define which {es} indices the rule searches for alerts.
.. Define which {es} indices or data view the rule searches for alerts.
.. Add an {ref}/eql-syntax.html[EQL statement] used to detect alerts.
+
For example, the following rule detects when `msxsl.exe` makes an outbound
Expand Down Expand Up @@ -216,12 +227,12 @@ NOTE: For sequence events, the {security-app} generates a single alert when all
[[create-indicator-rule]]
==== Create an indicator match rule

NOTE: {es-sec} provides limited support for indicator match rules. See <<support-indicator-rules>> for more information.
NOTE: {elastic-sec} provides limited support for indicator match rules. See <<support-indicator-rules>> for more information.

. To create an indicator match rule that searches for events whose specified field value matches the specified indicator field value in the indicator index patterns, select *Indicator Match*, then fill in the following fields:
.. *Index patterns*: The {es-sec} event indices on which the rule runs.
. To create a rule that searches for events whose specified field value matches the specified indicator field value in the indicator index patterns, select *Indicator Match*, then fill in the following fields:
.. *Source*: The individual index patterns or data view that specifies what data to search.
.. *Custom query*: The query and filters used to retrieve the required results from
the {es-sec} event indices. For example, if you want to match documents that only contain a `destination.ip` address field, add `destination.ip : *`.
the {elastic-sec} event indices. For example, if you want to match documents that only contain a `destination.ip` address field, add `destination.ip : *`.
+
TIP: If you want the rule to check every field in the indices, use this
wildcard expression: `*:*`.
Expand Down Expand Up @@ -255,6 +266,21 @@ field values.
+
. Continue with <<preview-rules, previewing the rule>> (optional) or click *Continue* to <<rule-ui-basic-params, configure basic rule settings>>.

[discrete]
[[create-new-terms-rule]]
==== Create a new terms rule
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved

. To create a rule that searches for each new term detected in source documents, select *New Terms*, then:
.. Specify what data to search by entering individual {es} index patterns or selecting an existing data view.
.. Use the filter and query fields to create the criteria used for detecting
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
alerts.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
+
NOTE: NOTE: You can use {kib} saved queries (image:images/saved-query-menu.png[Saved query menu,17,17]) and queries from saved Timelines (`Import query from saved Timeline`) as rule conditions.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
+
.. Use the `Fields` menu to select a field to check for new terms.
.. Use the `History Window Size` to specify the time range to search in minutes, hours, or days to determine if a term is new.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
..Continue with <<preview-rules, previewing the rule>> (optional) or click *Continue* to <<rule-ui-basic-params, configure basic rule settings>>.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved

[discrete]
[[preview-rules]]
=== Preview your rule (optional)
Expand Down Expand Up @@ -366,11 +392,10 @@ caused the event directly in the Alerts table.
+
NOTE: For threshold rules, not all source event values can be used for overrides; only the fields that were aggregated over (the `Group by` fields) will contain data.
.. *Timestamp override* (optional): Select a source event timestamp field. When selected, the rule's query uses the selected field, instead of the default `@timestamp` field, to search for alerts. This can help reduce missing alerts due to network or server outages. Specifically, if your ingest pipeline adds a timestamp when events are sent to {es}, this avoids missing alerts due to ingestion delays.
However, if you know your data source has an inaccurate `@timestamp` value, it is recommended you check the *Do not use @timestamp as a fallback timestamp field* option to ignore the `@timestamp` field entirely.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved
+
TIP: These Filebeat modules have an `event.ingested` timestamp field that can
be used instead of the default `@timestamp` field:
{filebeat-ref}/filebeat-module-microsoft.html[Microsoft] and
{filebeat-ref}/filebeat-module-google_workspace.html[Google Workspace].
TIP: The {filebeat-ref}/filebeat-module-microsoft.html[Microsoft] and
{filebeat-ref}/filebeat-module-google_workspace.html[Google Workspace] Filebeat modules have an `event.ingested` timestamp field that can be used instead of the default `@timestamp` field.
jmikell821 marked this conversation as resolved.
Show resolved Hide resolved

. Click *Continue*. The *Schedule rule* pane is displayed.
+
Expand Down