diff --git a/docs/detections/api/rules/rules-api-create.asciidoc b/docs/detections/api/rules/rules-api-create.asciidoc index 621d3fe102..1e38b2abd1 100644 --- a/docs/detections/api/rules/rules-api-create.asciidoc +++ b/docs/detections/api/rules/rules-api-create.asciidoc @@ -1064,7 +1064,7 @@ POST api/detection_engine/rules { "type": "esql", "language": "esql", - "query": "from auditbeat-8.10.2 [metadata _id, _version, _index] | where process.parent.name == \"EXCEL.EXE\"", + "query": "from auditbeat-8.10.2 METADATA _id, _version, _index | where process.parent.name == \"EXCEL.EXE\"", "name": "Find Excel events", "description": "Find Excel events", "tags": [], @@ -1525,7 +1525,7 @@ Example response for an {esql} rule: "setup": "", "type": "esql", "language": "esql", - "query": "from auditbeat-8.10.2 [metadata _id] | where process.parent.name == \"EXCEL.EXE\"" + "query": "from auditbeat-8.10.2 METADATA _id | where process.parent.name == \"EXCEL.EXE\"" } -------------------------------------------------- <1> dev:[] These fields are under development and their usage may change: `related_integrations` and `required_fields`. diff --git a/docs/detections/rules-ui-create.asciidoc b/docs/detections/rules-ui-create.asciidoc index bf8e964672..46bb76974e 100644 --- a/docs/detections/rules-ui-create.asciidoc +++ b/docs/detections/rules-ui-create.asciidoc @@ -311,16 +311,16 @@ NOTE: Rules that use aggregating queries might create duplicate alerts. This can [float] [[esql-non-agg-query]] ===== Non-aggregating query -Non-aggregating queries doesn't use `STATS...BY` functions and doesn't aggregate source event data. Alerts generated by an {esql} rule with a non-aggregating query only contain the fields returned by the query. +Non-aggregating queries don't use `STATS...BY` functions and don't aggregate source event data. Alerts generated by an {esql} rule with a non-aggregating query only contain the fields returned by the query. Here is an example non-aggregating query: [source,esql] ----- -FROM logs-* [metadata _id, _index, _version] +FROM logs-* METADATA _id, _index, _version | WHERE event.category == "process" AND event.id == "8a4f500d" | LIMIT 10 ----- -- This query starts by querying logs from indices that match the pattern `logs-*`. The `[metadata _id, _index, _version]` operator allows <>. +- This query starts by querying logs from indices that match the pattern `logs-*`. The `METADATA _id, _index, _version` operator allows <>. - Next, the query filters events where the `event.category` is a process and the `event.id` is `8a4f500d`. - Then, it limits the output to the top 10 results. @@ -328,11 +328,11 @@ FROM logs-* [metadata _id, _index, _version] [[esql-non-agg-query-dedupe]] ===== Turn on alert deduplication for rules using non-aggregating queries -To deduplicate alerts, a query needs access to the `_id`, `_index`, and `_version` metadata fields of the queried source event documents. You can allow this by adding the `[metadata _id, _index, _version]` operator after the `FROM` source command, for example: +To deduplicate alerts, a query needs access to the `_id`, `_index`, and `_version` metadata fields of the queried source event documents. You can allow this by adding the `METADATA _id, _index, _version` operator after the `FROM` source command, for example: [source,esql] ----- -FROM logs-* [metadata _id, _index, _version] +FROM logs-* METADATA _id, _index, _version | WHERE event.category == "process" AND event.id == "8a4f500d" | LIMIT 10 ----- @@ -345,7 +345,7 @@ Here is an example of a query that fails to deduplicate alerts. It uses the `DRO [source,esql] ----- -FROM logs-* [metadata _id, _index, _version] +FROM logs-* METADATA _id, _index, _version | WHERE event.category == "process" AND event.id == "8a4f500d" | DROP _id | LIMIT 10 @@ -355,7 +355,7 @@ Here is another example of an invalid query that uses the `KEEP` command to only [source,esql] ----- -FROM logs-* [metadata _id, _index, _version] +FROM logs-* METADATA _id, _index, _version | WHERE event.category == "process" AND event.id == "8a4f500d" | KEEP event.* | LIMIT 10 @@ -367,7 +367,7 @@ FROM logs-* [metadata _id, _index, _version] When writing your query, consider the following: -- The {ref}/esql-commands.html#esql-limit[`LIMIT`] command specifies the number of rows an {esql} query returns and the number of alerts created per rule execution. Similarly, a detection rule's <> setting specifies the maximum number of alerts it can create every time it runs. +- The {ref}/esql-commands.html#esql-limit[`LIMIT`] command specifies the maximum number of rows an {esql} query returns and the maximum number of alerts created per rule execution. Similarly, a detection rule's <> setting specifies the maximum number of alerts it can create every time it runs. + If the `LIMIT` value is lower than the `max_signals` value, the rule uses the `LIMIT` value to determine the maximum number of alerts the rule generates. If the `LIMIT` value is higher than the `max_signals` value, the rule uses the `max_signals` value. + @@ -382,7 +382,7 @@ NOTE: The `max_signals` default value is 100. You can modify it using the <>. +- If your {esql} query creates new fields that aren’t part of the ECS schema, they won’t be mapped to the alerts index, and you can't search or filter for them from the Alerts table. As a workaround, create <>. - If your {esql} query creates new fields that aren’t in the query’s source index, they can’t be added to the rule’s <>. [float]