[Custom threshold rule] Disable missing group for document count aggregation

[maryam-saeidi]

📝 Summary

In the Metric threshold rule, if the selected aggregation is document count, the missing group setting would be disabled with the following message:

[This setting is not applicable to the Document Count aggregator.]

Now, in the custom threshold rule, we don't have document count at the top level as custom aggregation is now the default one, so there are two questions to answer:

1. Why did we have this logic to disable missing group settings when selected aggregation is document count?
2. Do we need to apply a similar logic for the custom threshold rule? If yes, is it only applicable when we only have one aggregation, which is document count, or should we consider other scenarios as well?

@simianhacker Do you happen to know the answer to the above questions, or who might know about this logic?

✅ Acceptance Criteria

• Implement the missing group disable logic based on the decision

[elasticmachine]

Pinging @elastic/actionable-observability (Team: Actionable Observability)

[simianhacker]

Ugh... this was in place before I added the NO_DATA action group. I probably should have removed this behavior because it doesn't really make sense. Also, we should have thrown an error when people defined doc_count < 1 or doc_count == 0 since that's really the same thing as NO_DATA.

[maryam-saeidi]

But, based on our previous discussion, when the missing group is selected, we will no longer use the NO_DATA logic, so how does NO_DATA cover this scenario?

we should have thrown an error when people defined doc_count < 1 or doc_count == 0

Do you mean adding both FE and BE validation for this logic?

[simianhacker]

Remind me again, are we getting rid of NO_DATA when the group by is not configured?

If so... then we would allow for doc_count < 1, if not we should throw some kind of error to warn the user that NO_DATA will be trigger instead of ALERT when doc_count < 1.

For group by rules... doc_count < 1 will never trigger on the individual group because the group wouldn't be returned when the document count is ZERO.

NO_DATA should always take precedence over doc_count < 1 in every scenario and we should let the user know when they've configured both. doc_count < 1 will never trigger for a group by rule.

[elasticmachine]

Pinging @elastic/obs-ux-management-team (Team:obs-ux-management)

[mholttech]

Just wanted to throw a comment on here, we just found in the "Metric Threshold" rule that even though it says "[This setting is not applicable to the Document Count aggregator.]" and the checkboxes are greyed out we are getting No Data Alerts when doing a Document Count threshold with Aggregation. I know this discussion is about the new Custom Threshold rule but wanted to provide some feedback of current behavior vs documented behavior. (Note: We're seeing this on 8.10.4, haven't tested on 8.11 yet). We have a support open for this.

[maryam-saeidi]

@mholttech Thanks for pointing that out. We are working on this new rule (Custom threshold) as a replacement for the Metric threshold rule. We will use your input for the new rule, and in case you need a fix for the Metric threshold behavior, feel free to report a bug in Kibana.

[maryam-saeidi]

After discussing this ticket with @simianhacker, we decided this logic is unnecessary for the new rule.

We still need to consider adding more information in the documentation of this rule related to how No data and doc_count < 1 work as Chris mentioned here:

NO_DATA should always take precedence over doc_count < 1 in every scenario and we should let the user know when they've configured both. doc_count < 1 will never trigger for a group by rule.

[maryam-saeidi]

I've tested some scenarios related to both Custom threshold and Metric threshold, here is the outcome of those tests:

Scenario	Custom threshold	Metric threshold
With data	{ value: 6, trigger: false, bucketKey: { groupBy0: '*' } }
No data	{ value: null, trigger: false, bucketKey: { groupBy0: '*' } }	{ value: 0, warn: false, trigger: false, bucketKey: { groupBy0: '*' } }
With group
Missing group	{ trigger: false, value: null, bucketKey: { groupBy0: 'host-1' }}	{ trigger: false, warn: false, value: null, bucketKey: { groupBy0: 'host-1' }}

As you can see, the value only for no data in the Metric threshold is 0, so we cannot differentiate between no data and having 0 documents. This issue will not happen in the Custom threshold rule as the aggregation will return null in this case.