Skip scheduling actions for the alerts without scheduledActions #195948
+355
−119
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Resolves: #190258
As a result of #190258, we have found out that the odd behaviour happens when an existing alert is pushed above the max alerts limit by a new alert.
Scenario:
The rule type detects 4 alerts (
alert-1
,alert-2
,alert-3
,alert-4
),But reports only the first 3 as the max alerts limit is 3.
alert-2
becomes recovered, therefore the rule type reports 3 active (alert-1
,alert-3
,alert-4
), 1 recovered (alert-2
) alert.Alerts
alert-1
,alert-3
,alert-4
are saved in the task state.alert-2
becomes active again (the others are still active)Rule type reports 3 active alerts (
alert-1
,alert-2
,alert-3
)As a result, the action scheduler tries to schedule actions for
alert-1
,alert-3
,alert-4
as they are the existing alerts.But, since the rule type didn't report the
alert-4
it has no scheduled actions, therefore the action scheduler assumes it is recovered and tries to schedule a recovery action.This PR changes the actionScheduler to handle active and recovered alerts separately.
With this change, no action would be scheduled for an alert from previous run (exists in the task state) and isn't reported by the ruleTypeExecutor due to max-alerts-limit but it would be kept in the task state.