[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 #137122

banderror · 2022-07-25T21:22:51Z

Related to: elastic/detection-rules#2073 (comment), #89876

Summary

Here we regenerate the MITRE ATT&CK model in the code based on the official MITRE content:

we update to the version ATT&CK-v11.3 (see [Rule Tuning] Missing MITRE ATT&CK Mappings detection-rules#2073 (comment))
this corresponds to the https://raw.githubusercontent.com/mitre/cti/ATT&CK-v11.3/enterprise-attack/enterprise-attack.json content

Also, this PR fixes the model regeneration script (check the comment below).

elasticmachine · 2022-07-25T21:22:54Z

Pinging @elastic/security-detections-response (Team:Detections and Resp)

elasticmachine · 2022-07-25T21:22:54Z

Pinging @elastic/security-solution (Team: SecuritySolution)

banderror · 2022-07-25T21:28:07Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+    .filter(
+      (obj) =>
+        obj.type === 'attack-pattern' &&
+        (obj.x_mitre_is_subtechnique === false || obj.x_mitre_is_subtechnique === undefined)


This script didn't work out of the box and I have to add this condition to it:

|| obj.x_mitre_is_subtechnique === undefined

because it looks like the official metadata has been changed since we used the script last time, and now a lot of techniques don't have x_mitre_is_subtechnique field in them.

banderror · 2022-07-25T21:32:10Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

-  const tactic = tactics.find(
-    (tactic) => tactic.name === startCase(camelCase(technique.tactics[0]))
-  );
+  const tactic = tacticsData.find((tactic) => tactic.shortName === technique.tactics[0]);


Instead of finding the parent tactic by doing text transformations:

const tactic = tactics.find( (tactic) => tactic.name === startCase(camelCase(technique.tactics[0])) );

we can leverage the tactic's short name that's available in the source json metadata.

I think this is a more robust way to do that.

dplumlee

lgtm!

kibana-ci · 2022-07-26T11:44:57Z

💚 Build Succeeded

Metrics [docs]

Async chunks

Total size of all lazy-loaded chunks that will be downloaded as the user navigates the app

id	before	after	diff
`securitySolution`	5.5MB	5.5MB	+43.6KB

History

💚 Build #60621 succeeded 658267f3cbe6a5c70192dce03bad628c6e2af2d0
💔 Build #60614 failed 4b21b01bfb111dd66fa41dfb7af76be8b1fba793

To update your PR or re-run it, just comment with:
@elasticmachine merge upstream

cc @banderror

xcrzx · 2022-07-26T13:13:23Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+// Every release we should update the version of MITRE ATT&CK content and regenerate the model in our code.
+// This version must correspond to the one used for prebuilt rules in https://github.com/elastic/detection-rules.
+// This version is basically a tag on https://github.com/mitre/cti/tags, or can be a branch name like `master`.


Thanks for clarifying that 👍

@xcrzx Yeah, sure! Don't think this is enough though. Ideally, we need to make this responsibility visible in our GH board and checklists and have a recurring ticket for it or try to automate it somehow as suggested in #89876.

vitaliidm

thanks @banderror for improving extract script

Left a couple of comments

vitaliidm · 2022-07-26T14:36:54Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+  const tactics = mitreData
+    .filter((obj) => obj.type === 'x-mitre-tactic')
+    .reduce((acc, item) => {
+      const { id, reference } = getIdReference(item.external_references);


if id and reference are empty. Do we still want to add item in tactics array?

No, I don't think we want it. I have a bunch of concerns about reliability of this script. But I don't want to make any further changes in this PR to not accidentally break anything. Opening a ticket for follow-up changes would be a better option IMHO.

vitaliidm · 2022-07-26T14:59:30Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+      let tactics = [];
+      const { id, reference } = getIdReference(item.external_references);
+      if (item.kill_chain_phases != null && item.kill_chain_phases.length > 0) {
+        item.kill_chain_phases.forEach((tactic) => {
+          tactics = [...tactics, tactic.phase_name];
+        });
      }


looks like can be extracted to another method, as it identical to this

Let's address it as part of a follow-up ticket. I can create one.

vitaliidm · 2022-07-26T15:05:31Z

x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js

+    .filter((obj) => obj.type === 'x-mitre-tactic')
+    .reduce((acc, item) => {


nit: by moving filter condition inside reduce, could save one traverse through tactics

Let's address it as part of a follow-up ticket.

terrancedejesus

Great work and thank you for being so swift!

#151931) ## Summary Updates MITRE ATT&CK mappings to `v12.1`, see `detection-rules` repo update here: elastic/detection-rules#2422. Last update was to `v11.3` in #137122. To update, I modified https://github.com/elastic/kibana/blob/1a19148c1818b9af3b7735a0b6001bbb6bd8d7ba/x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js#L22 to point to the `ATT&CK-v12.1` tag. Then ran `yarn extract-mitre-attacks` from the root `security_solution` plugin directory, and then `node scripts/i18n_check.js --fix` from Kibana root to regen the i18n files. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md)

elastic#151931) ## Summary Updates MITRE ATT&CK mappings to `v12.1`, see `detection-rules` repo update here: elastic/detection-rules#2422. Last update was to `v11.3` in elastic#137122. To update, I modified https://github.com/elastic/kibana/blob/1a19148c1818b9af3b7735a0b6001bbb6bd8d7ba/x-pack/plugins/security_solution/scripts/extract_tactics_techniques_mitre.js#L22 to point to the `ATT&CK-v12.1` tag. Then ran `yarn extract-mitre-attacks` from the root `security_solution` plugin directory, and then `node scripts/i18n_check.js --fix` from Kibana root to regen the i18n files. ### Checklist - [X] Any text added follows [EUI's writing guidelines](https://elastic.github.io/eui/#/guidelines/writing), uses sentence case text and includes [i18n support](https://github.com/elastic/kibana/blob/master/packages/kbn-i18n/README.md) (cherry picked from commit bbfa43a)

…o v12.1 (#151931) (#152006) # Backport This will backport the following commits from `main` to `8.6`: - [[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 (#151931)](#151931)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Garrett Spong <[email protected]>

…o v12.1 (#151931) (#152007) # Backport This will backport the following commits from `main` to `8.7`: - [[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 (#151931)](#151931)  ### Questions ? Please refer to the [Backport tool documentation](https://github.com/sqren/backport)  Co-authored-by: Garrett Spong <[email protected]>

banderror requested a review from a team as a code owner July 25, 2022 21:22

banderror self-assigned this Jul 25, 2022

banderror requested a review from a team as a code owner July 25, 2022 21:22

banderror requested a review from vitaliidm July 25, 2022 21:22

banderror commented Jul 25, 2022

View reviewed changes

banderror requested review from dplumlee, terrancedejesus and Mikaayenson July 25, 2022 21:32

banderror mentioned this pull request Jul 25, 2022

[Rule Tuning] Missing MITRE ATT&CK Mappings elastic/detection-rules#2073

Merged

dplumlee approved these changes Jul 25, 2022

View reviewed changes

banderror added 3 commits July 26, 2022 12:35

Fix the script and regenerate the model

4241c4d

Remove unused keys from the localization dictionaries

c9f8c0c

Regenerate the model based on ATT&CK-v11.3

1bda53f

banderror force-pushed the update-mitre-attack-model branch from 658267f to 1bda53f Compare July 26, 2022 10:36

banderror changed the title ~~[Security Solution][Detections] Update the MITRE ATT&CK model~~ [Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 Jul 26, 2022

xcrzx approved these changes Jul 26, 2022

View reviewed changes

vitaliidm reviewed Jul 26, 2022

View reviewed changes

terrancedejesus approved these changes Jul 27, 2022

View reviewed changes

banderror merged commit cc634ed into elastic:main Jul 27, 2022

kibanamachine added the backport:skip This commit does not require backporting label Jul 27, 2022

banderror deleted the update-mitre-attack-model branch July 27, 2022 13:20

spong mentioned this pull request Feb 22, 2023

[Security Solution][Detections] Updates MITRE ATT&CK mappings to v12.1 #151931

Merged

1 task

banderror mentioned this pull request Sep 12, 2023

[Security Solution] Update MITRE ATT&CK mappings to the next version #166152

Open

6 tasks

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 #137122

[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 #137122

banderror commented Jul 25, 2022 •

edited

Loading

elasticmachine commented Jul 25, 2022

elasticmachine commented Jul 25, 2022

banderror Jul 25, 2022

banderror Jul 25, 2022

dplumlee left a comment

kibana-ci commented Jul 26, 2022

xcrzx Jul 26, 2022

banderror Jul 27, 2022 •

edited

Loading

vitaliidm left a comment

vitaliidm Jul 26, 2022

banderror Jul 27, 2022

vitaliidm Jul 26, 2022

banderror Jul 27, 2022

vitaliidm Jul 26, 2022

banderror Jul 27, 2022

terrancedejesus left a comment

		.filter((obj) => obj.type === 'x-mitre-tactic')
		.reduce((acc, item) => {

[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 #137122

[Security Solution][Detections] Update the MITRE ATT&CK model to v11.3 #137122

Conversation

banderror commented Jul 25, 2022 • edited Loading

Summary

elasticmachine commented Jul 25, 2022

elasticmachine commented Jul 25, 2022

Choose a reason for hiding this comment

Choose a reason for hiding this comment

dplumlee left a comment

Choose a reason for hiding this comment

kibana-ci commented Jul 26, 2022

💚 Build Succeeded

Metrics [docs]

Async chunks

History

Choose a reason for hiding this comment

banderror Jul 27, 2022 • edited Loading

Choose a reason for hiding this comment

vitaliidm left a comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

Choose a reason for hiding this comment

terrancedejesus left a comment

Choose a reason for hiding this comment

banderror commented Jul 25, 2022 •

edited

Loading

banderror Jul 27, 2022 •

edited

Loading