[Security Solution][Detection Engine] When sets of documents have identical timestamps, they can potentially be skipped during alert creation

[dplumlee]

Overview

During alert creation within the detection engine, sets of alerts that have the same timestamp can potentially be skipped over. This occurs if a set of alerts that have identical timestamps are present at the end of a search window, overlapping between the current and next pages. We use the search_after es parameter to paginate through the search results, but if there are any remaining alerts that have an identical timestamp remaining past the end of the search window, they will be skipped when retrieving the next result.

Example

[...]
{
  "@timestamp": "2023-08-17T20:53:35.462Z" // 98
},
{
  "@timestamp": "2023-08-17T20:53:35.462Z" // 99
},
-----search stops here-----
{
  "@timestamp": "2023-08-17T20:53:35.462Z" // 100 (skipped)
},
{
  "@timestamp": "2023-08-17T20:53:35.462Z" // 101 (skipped)
},
-----and will start again here-----
{
  "@timestamp": "2023-08-17T20:53:35.598Z" // 102
},
[...]

Possible solutions

A solution to this might be incorporating a PIT for the search to ensure events don't get skipped when paginating through search results.

Related occurrences

• https://discuss.elastic.co/t/security-alerts-not-generated-for-each-document/341049/4

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)