[EDR Workflows] Enable response actions in base rule params

[tomsonpl]

Another iteration (and the last one) of enabling automated response actions in rules - ML, Threshold, Indicator Match.

Initially we thought we couldn't support automated response_actions in all rule types. We were proven to be wrong, and happily are exposing response_actions to all rules now 🥳

This PR removes the optional response_actions parameter from type specific rules, and enable it as common in all rules.

The UI of the functionality is hidden behind the feature flag for purpose of intermediate release.
The feature flag and checks for ruleTypes will be removed in a following PR - which will result in all rule types being supported.

[tomsonpl]

/ci

[tomsonpl]

/ci

[tomsonpl]

/ci

[tomsonpl]

/ci

[tomsonpl]

/ci

[tomsonpl]

/ci

[tomsonpl]

/ci

[elasticmachine]

Pinging @elastic/security-defend-workflows (Team:Defend Workflows)

[pmuellr]

LGTM, given the following:

• adds optional parameter to some rule types
• will be rolled out with an "intermediate release" to allow rollback /zdt in serverless (as noted in PR top comment)

[paul-tavares]

Files impacting elastic/security-defend-workflows team LGTM 👍

[kibana-ci]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: b7ae028

Failed CI Steps

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	20.6MB	20.6MB	-405.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	88.3KB	88.3KB	+46.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
securitySolution	539	538	-1

Total ESLint disabled count

id	before	after	diff
securitySolution	624	623	-1

History

• 💔 Build #239806 failed d666b18
• 💔 Build #239780 failed b169902
• 💔 Build #239690 failed fed68e1
• 💛 Build #239423 was flaky 90e4ec5
• 💔 Build #239394 failed f65fb1c

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @tomsonpl

[maximpn]

@tomsonpl The changes look good and straightforward overall. Moving response_actions to BaseOptionalFields schema allows to simplify the implementation. My concern that not all rule types support response actions. shouldShowResponseActions() defines which rule type support them. Meantime API uses Zod schemas to verify accepted fields. It means that response_actions field will be accepted by all rule types and saved in a corresponding SO. It doesn't look like expected behavior. In that case route's implementation should be tweaked to clear sent response actions for unsupported rule types.

[maximpn]

Previously we discussed that only specific rule types support response actions. Adding it to base BaseOptionalFields schema will make it possible passing response_actions via rule CRUD API. Does it mean all rule types support response actions now?

[tomsonpl]

Thanks Maxim 👍
Yes, you got it right. And it's my fault that I brought the confusion. Previously we were more than sure that not all rules support agent.id in alerts, but apparently it is depending fully on the source of alert. We discussed this more in slack and I tested with more rules and apparently we can enable that in ALL rule types.

The shouldShowResponseActions is going to be there just for the first release of serverless (intermediate release) to have the logic on backend, but not expose response_actions in the UI of newly added types of rules. But after a week with another serverless release I am going to remove this check totally.

How does it sound?

[maximpn]

@tomsonpl Thanks for the explanation. I recommend adding to the PR description before merging it.

[elasticmachine]

💛 Build succeeded, but was flaky

• Buildkite Build
• Commit: a94110e

Failed CI Steps

• Jest Tests #11

Test Failures

• [job] [logs] Jest Tests #11 / query tab with unified timeline Leading actions - notes securitySolutionNotesEnabled = true should have the notification dot & correct tooltip

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
securitySolution	20.6MB	20.6MB	-405.0B

Page load bundle

Size of the bundles that are downloaded on every page load. Target size is below 100kb

id	before	after	diff
securitySolution	88.3KB	88.3KB	+46.0B

Unknown metric groups

ESLint disabled line counts

id	before	after	diff
securitySolution	539	538	-1

Total ESLint disabled count

id	before	after	diff
securitySolution	624	623	-1

cc @tomsonpl

[rylnd]

Detection Engine changes look good to me! No real concerns or objects, and I tested what I could with the feature flag.

[szwarckonrad]

DW changes LGTM!

[szwarckonrad]

Consider increasing readability by going, for example, this route:

export const shouldShowResponseActions = (
  ruleType: Type | undefined,
  automatedResponseActionsForAllRulesEnabled: boolean
): boolean => {
  const isEligibleRuleType = 
    isQueryRule(ruleType) || 
    isEsqlRule(ruleType) || 
    isEqlRule(ruleType) || 
    isNewTermsRule(ruleType);

  const isAutomatedResponseApplicable = 
    automatedResponseActionsForAllRulesEnabled && 
    (isThresholdRule(ruleType) || isThreatMatchRule(ruleType) || isMlRule(ruleType));

  return isEligibleRuleType || isAutomatedResponseApplicable;
};

[tomsonpl]

Yes, I agree 👍 this would increase readability. However, this check will be removed next week altogether with the feature flag, so I would prefer not to change it now, and just merge the thing :P
Are you ok with that? :)

[szwarckonrad]

Sure :)

[kibanamachine]

Starting backport for target branches: 8.x

https://github.com/elastic/kibana/actions/runs/11256680219

[kibanamachine]

💚 All backports created successfully

Status	Branch	Result
✅	8.x

Note: Successful backport PRs will be merged automatically after passing CI.

Questions ?

Please refer to the Backport tool documentation