[Security Solution] Prebuilt rule's exceptions are lost when the rule is upgraded to a new version in which the rule's type is different

[banderror]

Summary

The bug occurs when:

• The user installs a prebuilt rule X and adds some exceptions to it.
• The TRADE team changes the rule type of rule X in https://github.com/elastic/detection-rules and releases a new package with the updated rule version.
• The user upgrades rule X to this updated rule version in Security Solution.

As a result, any exceptions added to rule X by the user will be lost after the upgrade.

Expected behavior:

• Any exception lists attached to the rule (the default list or shared lists) are preserved in the rule after the upgrade.
• Any exception lists attached to the rule (the default list or shared lists) are preserved regardless of the type of the target version (whether the type has been changed or not, it shouldn't have any impact on the exception lists).

Todo

• Fix the bug
• Add scenario(s) to the test plan
• Add test coverage for the bugfix according to the test plan

Real example

This just happened to one of our users who upgraded the following prebuilt rule: https://www.elastic.co/guide/en/security/8.10/prebuilt-rule-8-10-4-enumeration-of-privileged-local-groups-membership.html

Indeed, the rule type of this rule was changed recently as you can see from commits: https://github.com/elastic/detection-rules/commits/9f41c9f35cd568c397c1f81bafc997e92bbf2df0/rules/windows/discovery_privileged_localgroup_membership.toml

Before upgrade:

After upgrade:

[elasticmachine]

Pinging @elastic/security-solution (Team: SecuritySolution)

[elasticmachine]

Pinging @elastic/security-detections-response (Team:Detections and Resp)

[terrancedejesus]

List of rules that have had query changes.

Query Type rules:

Rule ID	Name	Version	Query Language	Type	Platform
0d69150b-96f8-467c-a86d-a67a3378ce77	Nping Process Activity	105	eql	eql	Endpoint:Linux
291a0de9-937a-4189-94c0-3e847c8b13e4	Enumeration of Privileged Local Groups Membership	208	kuery	new_terms	Endpoint:Windows
2d8043ed-5bda-4caf-801c-c1feb7410504	Enumeration of Kernel Modules	205	kuery	new_terms	Endpoint:Linux
2f8a1226-5720-437d-9c20-e0029deb6194	Attempt to Disable Syslog Service	106	eql	eql	Endpoint:Linux
852c1f19-68e8-43a6-9dce-340771fe1be3	Suspicious PowerShell Engine ImageLoad	208	kuery	new_terms	Endpoint:Windows
90169566-2260-4824-b8e4-8615c3b4ed52	Hping Process Activity	105	eql	eql	Endpoint:Linux
9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae2	Microsoft Build Engine Started by a Script Process	205	kuery	new_terms	Endpoint:Windows
9d110cb3-5f4b-4c9a-b9f5-53f0a1707ae6	Microsoft Build Engine Started an Unusual Process	206	kuery	new_terms	Endpoint:Windows
9f9a2a82-93a8-4b1a-8778-1780895626d4	File Permission Modification in Writable Directory	105	eql	eql	Endpoint:Linux
a00681e3-9ed6-447c-ab2c-be648821c622	First Time Seen AWS Secret Value Accessed in Secrets Manager	207	kuery	new_terms	AWS
cac91072-d165-11ec-a764-f661ea17fbce	Abnormal Process ID or Lock File Created	209	kuery	new_terms	Endpoint:Linux
cd66a5af-e34b-4bb0-8931-57d0a043f2ef	Kernel Module Removal	105	eql	eql	Endpoint:Linux
d76b02ef-fc95-4001-9297-01cb7412232f	Interactive Terminal Spawned via Python	107	eql	eql	Endpoint:Linux
fd7a6052-58fa-4397-93c3-4795249ccfa2	Svchost spawning Cmd	207	kuery	new_terms	Endpoint:Windows

[maximpn]

Checking the code it's not so hard to find upgradeRule() function which performs prebuilt rule upgrade. It has the following comment

If we're trying to change the type of a prepackaged rule, we need to delete the old one and replace it with the new rule, keeping the enabled setting, actions, throttle, id, and exception lists from the old rule.

Looking below in the code it's clear that only enabled state and actions get restored upon rule upgrade. Missing to restore exceptions lists leads to disappearing exceptions upon rule upgrade.

I made testing and discovered that execution results and execution events also get lost due to missing to restore saved object id. Execution log isn't gone anywhere but can't be bound to a new id. Direct links to rule details page won't work neither after upgrade.

We may need to restore another fields as well. This ticket contains fields list and might help to anylize what should be preserved. As minimum execution_summary and outcome cleared out after upgrade.

@banderror @approksiu do you have in mind the other fields (besides exceptionsLists and id) we must preserve?

[approksiu]

@maximpn do we have any fields related to actions and their settings? those should stay.
Do we keep history of alerts after such an upgrade?

[maximpn]

@approksiu enabled state, actions and alerts don't disappear after upgrade.

[approksiu]

@maximpn I don't see other fields we should preserve.

[banderror]

Had a chat on that today with Maxim and also read the comment carefully. @maximpn I have nothing to add.

For transparency, fields to preserve we talked about:

• The list of exception list ids attached to the rule
• Actions with all their parameters
• Rule id (saved object's id) and rule_id
• Timeline template id and title
• Array of saved object references (this is a core saved object's property), if it's possible to recreate it in the new rule

Fields we can drop:

• execution_summary and the internal rule fields that are used to create it. Loosing it will lead to resetting the last rule execution status to unknown, and it shouldn't be a big deal

[maximpn]

Thank you for the comment @banderror 👍

I've checked the rules client and currently there is no such an option to specify saved object references.

The docs say the following about this field

Objects with name, id, and type properties that describe the other saved objects this object references. To refer to the other saved object, use name in the attributes, but never the id, which automatically updates during migrations or import/export.

I played around and noticed only exception SO references are saved in the field

"references": [
  {
    "name": "param:exceptionsList_0",
    "id": "0a3c4d42-97c7-4ed8-a52c-5459a8035d7a",
    "type": "exception-list"
   },
   {
     "name": "param:exceptionsList_1",
     "id": "d2620ba8-e09b-4f4c-a545-ea5b0afddf6a",
     "type": "exception-list"
    }
]

In this case it's not necessary to restore this field because we pass exceptions list via the API and the field will be filled in by the SO client under the hood.

[banderror]

In this case it's not necessary to restore this field because we pass exceptions list via the API and the field will be filled in by the SO client under the hood.

@maximpn Can you elaborate on that please, not sure I got when this happens and what API do you mean.

[maximpn]

@banderror Sure. I omitted some details for simplicity but let me explain.

createRules function is used while upgrading a prebuilt rule. The same createRules function is used when a rule is created via rule management API.

By passing exceptionsList to createRules() we say this rule has some exceptions. Under the hood exceptions are extracted into SO references. And the same happens when updating or patching a rule.

I also doubled checked via create rule API. Exceptions are in place after all manipulations as expected.

You can check out extractReferences() on Security Solution side used in createSecurityRuleTypeWrapper() and createRule() at Alerting framework side for more details.

[maximpn]

@vgomez-el Could you please validate the fix? It has been backported to 8.12 and will be released in 8.12.2.

[vgomez-el]

I have deployed a 8.10.4 version and activated the Query type prebuilt rules that @terrancedejesus mentioned on his comment

REC-20240301182245.mp4

and after upgrading the version to 8.12.2 rules are updated and they keep their exceptions. So we can consider the bug fixed and validated for 8.12.2:

REC-20240301184426.mp4

Thanks @maximpn and @banderror and the rest of people who contributed to fix this bug!