[Security Solution] [Detections] [Platform] Expose _tier in the Security Solution

[dhurley14]

Currently (versions <= 8.1), if users want to segment data in the security solution between the different data tiers they have to rely on index aliases. By exposing the _tier field in queries executed in the security solution, we can better provide users with finer-grained controls when searching for alerts and associated source events from hot,warm,cold,frozen nodes.

This issue will serve as a reference as work progresses towards this goal.

TODO:

1. Update the UI for rules to accept tiers from which to include in the given rules execution
2. Update sourcerer to include a tier selection for queries executed against a given data view.

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[yctercero]

Note that @banderror mentioned this issue here #158495

Closed that one as duplicate.

[s-abdelwahhab]

Any updates about this please?

[yctercero]

Hi @s-abdelwahhab ! When we were initially discussing this issue we were trying to figure out if adding a _tier filter could help avoid EQL query rules from querying frozen tiers during the pre-search phase. It was determined that such a filter, even if exposed, would not be applied in that instance.

If you're able to, would love to understand your use case here.

[marshallmain]

Customer in https://discuss.elastic.co/t/coordinating-nodes-high-circuit-breaker-tripped-counts/344161/11 also suggested an option to filter on _tier to deal with queries hitting the frozen tier due to future timestamps. In the thread, a number of prebuilt new terms and EQL sequence rules time out due to frozen tier nodes and future timestamps. Disabling fallback to @timestamp fixes the issue, but a filter on _tier also improves performance without having to disable fallback.