diff --git a/docs/AI-for-security/llm-performance-matrix.asciidoc b/docs/AI-for-security/llm-performance-matrix.asciidoc index c8f9e845c3..abed9dfdb2 100644 --- a/docs/AI-for-security/llm-performance-matrix.asciidoc +++ b/docs/AI-for-security/llm-performance-matrix.asciidoc @@ -13,4 +13,5 @@ This table describes the performance of various large language models (LLMs) for | *Assistant - Knowledge retrieval* | Good | Excellent | Excellent | Excellent | Excellent | Excellent | Great | Excellent | Excellent | *Attack Discovery* | Great | Great | Excellent | Poor | Poor | Great | Poor | Excellent | Poor |=== - \ No newline at end of file + +NOTE: `Excellent` is the best rating, followed by `Great`, then by `Good`, and finally by `Poor`. \ No newline at end of file diff --git a/docs/cloud-native-security/cloud-native-security-index.asciidoc b/docs/cloud-native-security/cloud-native-security-index.asciidoc index 742149aa26..935c121a8b 100644 --- a/docs/cloud-native-security/cloud-native-security-index.asciidoc +++ b/docs/cloud-native-security/cloud-native-security-index.asciidoc @@ -41,6 +41,7 @@ include::cspm.asciidoc[leveloffset=+1] include::cspm-get-started-aws.asciidoc[leveloffset=+2] include::cspm-get-started-gcp.asciidoc[leveloffset=+2] include::cspm-get-started-azure.asciidoc[leveloffset=+2] +include::cspm-permissions.asciidoc[leveloffset=+2] include::cspm-findings.asciidoc[leveloffset=+2] include::cspm-benchmark-rules.asciidoc[leveloffset=+2] include::cspm-cloud-posture-dashboard.asciidoc[leveloffset=+2] diff --git a/docs/cloud-native-security/cspm-get-started-aws.asciidoc b/docs/cloud-native-security/cspm-get-started-aws.asciidoc index 9ac8268747..4bc8f107a7 100644 --- a/docs/cloud-native-security/cspm-get-started-aws.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-aws.asciidoc @@ -10,17 +10,10 @@ This page explains how to get started monitoring the security posture of your cl .Requirements [sidebar] -- +* Minimum privileges vary depending on whether you need to read, write, or manage CSPM data and integrations. Refer to <>. * The CSPM integration is available to all {ecloud} users. On-premise deployments require an https://www.elastic.co/pricing[Enterprise subscription]. * CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. * CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported. https://github.com/elastic/kibana/issues/new/choose[Click here to request support]. -* `Read` privileges for the following {es} indices: -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -* The following {kib} privileges: -** Security: `Read` -** Integrations: `Read` -** Saved Objects Management: `Read` -** Fleet: `All` * The user who gives the CSPM integration AWS permissions must be an AWS account `admin`. -- diff --git a/docs/cloud-native-security/cspm-get-started-azure.asciidoc b/docs/cloud-native-security/cspm-get-started-azure.asciidoc index 865ebf02b0..4e78781323 100644 --- a/docs/cloud-native-security/cspm-get-started-azure.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-azure.asciidoc @@ -10,17 +10,10 @@ This page explains how to get started monitoring the security posture of your cl .Requirements [sidebar] -- +* Minimum privileges vary depending on whether you need to read, write, or manage CSPM data and integrations. Refer to <>. * The CSPM integration is available to all {ecloud} users. On-premise deployments require an https://www.elastic.co/pricing[Enterprise subscription]. * CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. * CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported. https://github.com/elastic/kibana/issues/new/choose[Click here to request support]. -* `Read` privileges for the following {es} indices: -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -* The following {kib} privileges: -** Security: `Read` -** Integrations: `Read` -** Saved Objects Management: `Read` -** Fleet: `All` * The user who gives the CSPM integration permissions in Azure must be an Azure subscription `admin`. -- diff --git a/docs/cloud-native-security/cspm-get-started-gcp.asciidoc b/docs/cloud-native-security/cspm-get-started-gcp.asciidoc index 30d34c74c0..dc5bfca23b 100644 --- a/docs/cloud-native-security/cspm-get-started-gcp.asciidoc +++ b/docs/cloud-native-security/cspm-get-started-gcp.asciidoc @@ -10,17 +10,10 @@ This page explains how to get started monitoring the security posture of your GC .Requirements [sidebar] -- +* Minimum privileges vary depending on whether you need to read, write, or manage CSPM data and integrations. Refer to <>. * The CSPM integration is available to all {ecloud} users. On-premise deployments require an https://www.elastic.co/pricing[Enterprise subscription]. * CSPM only works in the `Default` {kib} space. Installing the CSPM integration on a different {kib} space will not work. * CSPM is supported only on AWS, GCP, and Azure commercial cloud platforms, and AWS GovCloud. Other government cloud platforms are not supported. https://github.com/elastic/kibana/issues/new/choose[Click here to request support]. -* `Read` privileges for the following {es} indices: -** `logs-cloud_security_posture.findings_latest-*` -** `logs-cloud_security_posture.scores-*` -* The following {kib} privileges: -** Security: `Read` -** Integrations: `Read` -** Saved Objects Management: `Read` -** Fleet: `All` * The user who gives the CSPM integration GCP permissions must be a GCP project `admin`. -- diff --git a/docs/cloud-native-security/cspm-permissions.asciidoc b/docs/cloud-native-security/cspm-permissions.asciidoc new file mode 100644 index 0000000000..c79a6fd36c --- /dev/null +++ b/docs/cloud-native-security/cspm-permissions.asciidoc @@ -0,0 +1,61 @@ +[[cspm-required-permissions]] += CSPM privilege requirements + +This page lists required privileges for {elastic-sec}'s CSPM features. There are three access levels: read, write, and manage. Each access level and its requirements are described below. + +[discrete] +== Read + +Users with these minimum permissions can view data on the **Findings** page and the Cloud Posture dashboard. + +[discrete] +=== {es} index privileges +`Read` privileges for the following {es} indices: + +* `logs-cloud_security_posture.findings_latest-*` +* `logs-cloud_security_posture.scores-*` + +[discrete] +=== {kib} privileges + +* `Security: Read` + + +[discrete] +== Write + +Users with these minimum permissions can view data on the **Findings** page and the Cloud Posture dashboard, create detection rules from the findings details flyout, and enable or disable benchmark rules. + +[discrete] +=== {es} index privileges +`Read` privileges for the following {es} indices: + +* `logs-cloud_security_posture.findings_latest-*` +* `logs-cloud_security_posture.scores-*` + +[discrete] +=== {kib} privileges + +* `Security: All` + + +[discrete] +== Manage + +Users with these minimum permissions can view data on the **Findings** page and the Cloud Posture dashboard, create detection rules from the findings details flyout, enable or disable benchmark rules, and install, update, or uninstall CSPM integrations and assets. + +[discrete] +=== {es} index privileges +`Read` privileges for the following {es} indices: + +* `logs-cloud_security_posture.findings_latest-*` +* `logs-cloud_security_posture.scores-*` + +[discrete] +=== {kib} privileges + +* `Security: All` +* `Spaces: All` +* `Fleet: All` +* `Integrations: All` + diff --git a/docs/cloud-native-security/environment-variable-capture.asciidoc b/docs/cloud-native-security/environment-variable-capture.asciidoc index ec05a561b8..36ecbd0f89 100644 --- a/docs/cloud-native-security/environment-variable-capture.asciidoc +++ b/docs/cloud-native-security/environment-variable-capture.asciidoc @@ -28,9 +28,6 @@ To set up environment variable capture for an {agent} policy: . Enter the names of env vars you want to capture, separated by commas. For example: `PATH,USER` . Click *Save*. -[role="screenshot"] -image::images/env-var-capture.png[The "linux.advanced.capture_env_vars" advanced agent policy setting] - [[find-cap-env-vars]] [discrete] == Find captured environment variables diff --git a/docs/cloud-native-security/images/env-var-capture.png b/docs/cloud-native-security/images/env-var-capture.png deleted file mode 100644 index d62ca4149c..0000000000 Binary files a/docs/cloud-native-security/images/env-var-capture.png and /dev/null differ diff --git a/docs/detections/detection-engine-intro.asciidoc b/docs/detections/detection-engine-intro.asciidoc index b522d3f4bd..105e2ade3e 100644 --- a/docs/detections/detection-engine-intro.asciidoc +++ b/docs/detections/detection-engine-intro.asciidoc @@ -167,3 +167,9 @@ and you should contact your {kib} administrator. NOTE: For *self-managed* {stack} deployments only, this message may be displayed when the <> setting is not enabled in the `elasticsearch.yml` file. For more information, refer to <>. + +[discrete] +[[detections-logsdb-index-mode]] +== Using logsdb index mode + +To learn how your rules and alerts are affected by using the {ref}/logs-data-stream.html[logsdb index mode], refer to <>. \ No newline at end of file diff --git a/docs/detections/detections-index.asciidoc b/docs/detections/detections-index.asciidoc index 2d1cab74d6..ff45aa827b 100644 --- a/docs/detections/detections-index.asciidoc +++ b/docs/detections/detections-index.asciidoc @@ -2,6 +2,8 @@ include::detection-engine-intro.asciidoc[] include::detections-req.asciidoc[leveloffset=+1] +include::detections-logsdb-impact.asciidoc[leveloffset=+1] + include::about-rules.asciidoc[] diff --git a/docs/detections/detections-logsdb-impact.asciidoc b/docs/detections/detections-logsdb-impact.asciidoc new file mode 100644 index 0000000000..1b304c7f1e --- /dev/null +++ b/docs/detections/detections-logsdb-impact.asciidoc @@ -0,0 +1,65 @@ +[[detections-logsdb-index-mode-impact]] += Using logsdb index mode with {elastic-sec} + +NOTE: To use the {ref}/mapping-source-field.html#synthetic-source[synthetic `_source`] feature, you must have the appropriate subscription. Refer to the subscription page for https://www.elastic.co/subscriptions/cloud[Elastic Cloud] and {subscriptions}[Elastic Stack/self-managed] for the breakdown of available features and their associated subscription tiers. + +This topic explains the impact of using logsdb index mode with {elastic-sec}. + +With logsdb index mode, the original `_source` field is not stored in the index but can be reconstructed using {ref}/mapping-source-field.html#synthetic-source[synthetic `_source`]. + +When the `_source` is reconstructed, {ref}/mapping-source-field.html#synthetic-source-modifications[modifications] are possible. Therefore, there could be a mismatch between users' expectations and how fields are formatted. + +Continue reading to find out how this affects specific {elastic-sec} components. + +[discrete] +[[logsdb-alerts]] +== Alerts + +When alerts are generated, the `_source` event is copied into the alert to retain the original data. When the logsdb index mode is applied, the `_source` event stored in the alert is reconstructed using synthetic `_source`. + +If you're switching to use logsdb index mode, the `_source` field stored in the alert might look different in certain situations: + +* {ref}/mapping-source-field.html#synthetic-source-modifications-leaf-arrays[Arrays can be reconstructed differently or deduplicated] +* {ref}/mapping-source-field.html#synthetic-source-modifications-field-names[Field names] +* `geo_point` data fields (refer to {ref}/mapping-source-field.html#synthetic-source-modifications-ranges[Representation of ranges] and {ref}/mapping-source-field.html#synthetic-source-precision-loss-for-point-types[Reduced precision of `geo_point` values] for more information) + +Alerts generated by the following rule types could be affected: + +* Custom query +* Event correlation (non-sequence only) +* Non-aggregate rule types (for example, {esql} rules that use non-aggregating queries) + +Alerts that are generated by threshold, {ml}, and event correlation sequence rules are not affected since they do not contain copies of the original source. + +[discrete] +[[logsdb-rule-actions]] +== Rule actions + +While we do not recommend using `_source` for actions, in cases where the action relies on the `_source`, the same limitations and changes apply. + +If you send alert notifications by enabling {kibana-ref}/alerting-getting-started.html#alerting-concepts-actions[actions] to the external systems that have workflows or automations based on fields formatted from the original source, they may be affected. In particular, this can happen when the fields used are arrays of objects. + +We recommend checking and adjusting the rule actions using `_source` before switching to logsdb index mode. + +[discrete] +[[logsdb-runtime-fields]] +== Runtime fields + +Runtime fields that reference `_source` may be affected. Some runtime fields might not work and need to be adjusted. For example, if an event was indexed with the value of `agent.name` in the dot-notation form, it will be returned in the nested form and might not work. + +The following is an example of accessing `_source` that works with the logsdb index mode enabled: + +[source,console] +---- +"source": """ emit(params._source.agent.name + "_____" + doc['agent.name'].value ); """ +"source": """ emit(params._source['agent']['name'] + "_____" + doc['agent.name'].value ); """ +"source": """ emit(field('agent.name').get(null) + "_____" + doc['agent.name'].value ); """ +"source": """ emit($('agent.name', null) + "_____" + doc['agent.name'].value ); """ +---- + +The following will not work with synthetic source (logsdb index mode enabled): + +[source,console] +---- +"source": """ emit(params._source['agent.name'] + "_____" + doc['agent.name'].value ); """ +---- \ No newline at end of file diff --git a/docs/release-notes.asciidoc b/docs/release-notes.asciidoc index c60a100248..49594201f1 100644 --- a/docs/release-notes.asciidoc +++ b/docs/release-notes.asciidoc @@ -3,6 +3,7 @@ This section summarizes the changes in each release. +* <> * <> * <> * <> @@ -69,6 +70,7 @@ This section summarizes the changes in each release. * <> * <> +include::release-notes/8.17.asciidoc[] include::release-notes/8.16.asciidoc[] include::release-notes/8.15.asciidoc[] include::release-notes/8.14.asciidoc[] diff --git a/docs/release-notes/8.16.asciidoc b/docs/release-notes/8.16.asciidoc index 3c062a314e..2a47e463cb 100644 --- a/docs/release-notes/8.16.asciidoc +++ b/docs/release-notes/8.16.asciidoc @@ -9,6 +9,63 @@ [[known-issue-8.16.1]] ==== Known issues +// tag::known-issue[201820] +[discrete] +.The **Exceptions** tab won't properly load if exceptions contain comments with newline characters (`\n`) +[%collapsible] +==== +*Details* + +On December 5, 2024, it was discovered that the **Exceptions** tab won't load properly if any exceptions contain comments with newline characters (`\n`). This issue occurs when you upgrade to 8.16.0 or later ({kibana-issue}201820[#201820]). + +*Workaround* + + +For custom rules: + +. From the **Rules** page, <> the rule or rules with the affected exception lists. +. Modify the `.ndjson` file so `comments` no longer contain newline characters. +. Return to the **Rules** page and <> the rules. Make sure to select the **Overwrite existing exception lists with conflicting "list_id"** option. + +For prebuilt rules: + +NOTE: If you only need to fix exceptions for the Elastic Endpoint rule, you can export and re-import its exception list from the <> page. + +. Follow these steps to fetch the affected exception list ID or IDs that are associated with the rule: +.. Find the affected rule's ID (`id`). From the **Rules** page, open the details of a rule, go to the page URL, and copy the string at the end. For example, in the URL http://host.name/app/security/rules/id/167a5f6f-2148-4792-8226-b5e7a58ef46e, the string at the end (`167a5f6f-2148-4792-8226-b5e7a58ef46e`) is the `id`. +.. Specify the `id` when fetching the rule's details using the {api-kibana}/operation/operation-readrule[Retrieve a detection rule API]. Here is an example request that includes the `id`: ++ +[source,console] +---- +curl -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' KIBANA_URL/api/detection_engine/rules?id=167a5f6f-2148-4792-8226-b5e7a58ef46e +---- ++ +.. The JSON response contains the `id`, `list_id`, and `namespace_type` values within the `exceptions_list` key (as shown below). You need these values when using the Exception list API to retrieve the affected exception list. ++ +[source,console] +---- +{ + "id": "167a5f6f-2148-4792-8226-b5e7a58ef46e", + "exceptions_list": [ + { + "id": "490525a2-eb66-4320-95b5-88bdd1302dc4", + "list_id": "f75aae6f-0229-413f-881d-81cb3abfbe2d", + "namespace_type": "single" + } + ] +} +---- ++ +. Use the export exceptions API to retrieve the affected exception list. Insert the values for the `id`, `list_id`, and `namespace_type` parameters into the following API call: ++ +[source,console] +---- +curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'KIBANA_URL/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson +---- ++ +. Modify the exception list's `.ndjson` file to ensure `comments[].comment` values don't contain newline characters (`\n`). +. Re-import the modified exception list using **Import exception lists** option on the <> page. The import will initially fail because the exception list already exists, and an option to overwrite the existing list will appear. Select the option, then resubmit the request to import the corrected exception list. +==== +// end::known-issue[201820] + // tag::known-issue[] [discrete] .Duplicate alerts can be produced from manually running threshold rules @@ -50,6 +107,63 @@ On November 12, 2024, it was discovered that manually running a custom query rul [[known-issue-8.16.0]] ==== Known issues +// tag::known-issue[201820] +[discrete] +.The **Exceptions** tab won't properly load if exceptions contain comments with newline characters (`\n`) +[%collapsible] +==== +*Details* + +On December 5, 2024, it was discovered that the **Exceptions** tab won't load properly if any exceptions contain comments with newline characters (`\n`). This issue occurs when you upgrade to 8.16.0 or later ({kibana-issue}201820[#201820]). + +*Workaround* + + +For custom rules: + +. From the **Rules** page, <> the rule or rules with the affected exception lists. +. Modify the `.ndjson` file so `comments` no longer contain newline characters. +. Return to the **Rules** page and <> the rules. Make sure to select the **Overwrite existing exception lists with conflicting "list_id"** option. + +For prebuilt rules: + +NOTE: If you only need to fix exceptions for the Elastic Endpoint rule, you can export and re-import its exception list from the <> page. + +. Follow these steps to fetch the affected exception list ID or IDs that are associated with the rule: +.. Find the affected rule's ID (`id`). From the **Rules** page, open the details of a rule, go to the page URL, and copy the string at the end. For example, in the URL http://host.name/app/security/rules/id/167a5f6f-2148-4792-8226-b5e7a58ef46e, the string at the end (`167a5f6f-2148-4792-8226-b5e7a58ef46e`) is the `id`. +.. Specify the `id` when fetching the rule's details using the {api-kibana}/operation/operation-readrule[Retrieve a detection rule API]. Here is an example request that includes the `id`: ++ +[source,console] +---- +curl -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' KIBANA_URL/api/detection_engine/rules?id=167a5f6f-2148-4792-8226-b5e7a58ef46e +---- ++ +.. The JSON response contains the `id`, `list_id`, and `namespace_type` values within the `exceptions_list` key (as shown below). You need these values when using the Exception list API to retrieve the affected exception list. ++ +[source,console] +---- +{ + "id": "167a5f6f-2148-4792-8226-b5e7a58ef46e", + "exceptions_list": [ + { + "id": "490525a2-eb66-4320-95b5-88bdd1302dc4", + "list_id": "f75aae6f-0229-413f-881d-81cb3abfbe2d", + "namespace_type": "single" + } + ] +} +---- ++ +. Use the export exceptions API to retrieve the affected exception list. Insert the values for the `id`, `list_id`, and `namespace_type` parameters into the following API call: ++ +[source,console] +---- +curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'KIBANA_URL/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson +---- ++ +. Modify the exception list's `.ndjson` file to ensure `comments[].comment` values don't contain newline characters (`\n`). +. Re-import the modified exception list using **Import exception lists** option on the <> page. The import will initially fail because the exception list already exists, and an option to overwrite the existing list will appear. Select the option, then resubmit the request to import the corrected exception list. +==== +// end::known-issue[201820] + // tag::known-issue[] [discrete] .Attempting to edit an Elastic AI Assistant Knowledge Base index results in an error diff --git a/docs/release-notes/8.17.asciidoc b/docs/release-notes/8.17.asciidoc new file mode 100644 index 0000000000..0dc2ba58fc --- /dev/null +++ b/docs/release-notes/8.17.asciidoc @@ -0,0 +1,129 @@ +[[release-notes-header-8.17.0]] +== 8.17 + +[discrete] +[[release-notes-8.17.0]] +=== 8.17.0 + +[discrete] +[[known-issue-8.17.0]] +==== Known issues + +// tag::known-issue[201820] +[discrete] +.The **Exceptions** tab won't properly load if exceptions contain comments with newline characters (`\n`) +[%collapsible] +==== +*Details* + +On December 5, 2024, it was discovered that the **Exceptions** tab won't load properly if any exceptions contain comments with newline characters (`\n`). This issue occurs when you upgrade to 8.16.0 or later ({kibana-issue}201820[#201820]). + +*Workaround* + + +For custom rules: + +. From the **Rules** page, <> the rule or rules with the affected exception lists. +. Modify the `.ndjson` file so `comments` no longer contain newline characters. +. Return to the **Rules** page and <> the rules. Ensure you select the **Overwrite existing exception lists with conflicting "list_id"** option. + +For prebuilt rules: + +NOTE: If you only need to fix exceptions for the Elastic Endpoint rule, you can export and re-import its exception list from the <> page. + +. Follow these steps to fetch the affected exception list ID or IDs that are associated with the rule: +.. Find the affected rule's ID (`id`). From the **Rules** page, open the details of a rule, go to the page URL, and copy the string at the end. For example, in the URL http://host.name/app/security/rules/id/167a5f6f-2148-4792-8226-b5e7a58ef46e, the string at the end (`167a5f6f-2148-4792-8226-b5e7a58ef46e`) is the `id`. +.. Specify the `id` when fetching the rule's details using the {api-kibana}/operation/operation-readrule[Retrieve a detection rule API]. Here is an example request that includes the `id`: ++ +[source,console] +---- +curl -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' KIBANA_URL/api/detection_engine/rules?id=167a5f6f-2148-4792-8226-b5e7a58ef46e +---- ++ +.. The JSON response contains the `id`, `list_id`, and `namespace_type` values within the `exceptions_list` key (as shown below). You need these values when using the Exception list API to retrieve the affected exception list. ++ +[source,console] +---- +{ + "id": "167a5f6f-2148-4792-8226-b5e7a58ef46e", + "exceptions_list": [ + { + "id": "490525a2-eb66-4320-95b5-88bdd1302dc4", + "list_id": "f75aae6f-0229-413f-881d-81cb3abfbe2d", + "namespace_type": "single" + } + ] +} +---- ++ +. Use the export exceptions API to retrieve the affected exception list. Insert the values for the `id`, `list_id`, and `namespace_type` parameters into the following API call: ++ +[source,console] +---- +curl -XPOST -H 'Authorization: ApiKey API_KEY_HERE' -H 'kbn-xsrf: true' -H 'elastic-api-version: 2023-10-31' 'KIBANA_URL/api/exception_lists/_export?list_id=f75aae6f-0229-413f-881d-81cb3abfbe2d&id=490525a2-eb66-4320-95b5-88bdd1302dc4&namespace_type=single' -o list.ndjson +---- ++ +. Modify the exception list's `.ndjson` file to ensure `comments[].comment` values don't contain newline characters (`\n`). +. Re-import the modified exception list using **Import exception lists** option on the <> page. The import will initially fail because the exception list already exists, and an option to overwrite the existing list will appear. Select the option, then resubmit the request to import the corrected exception list. +==== +// end::known-issue[201820] + +// tag::known-issue[] +[discrete] +.Duplicate alerts can be produced from manually running threshold rules +[%collapsible] +==== +*Details* + +On November 12, 2024, it was discovered that manually running threshold rules could produce duplicate alerts if the date range was already covered by a scheduled rule execution. + +==== +// end::known-issue[] + +// tag::known-issue[] +[discrete] +.Manually running custom query rules with suppression could suppress more alerts than expected +[%collapsible] +==== +*Details* + +On November 12, 2024, it was discovered that manually running a custom query rule with suppression could incorrectly inflate the number of suppressed alerts. + +==== +// end::known-issue[] + +[discrete] +[[features-8.17.0]] +==== New features +* Adds a signature option for trusted applications on macOS ({kibana-pull}197821[#197821]). +* Adds GA support for the case action feature, which lets rules automatically create cases ({kibana-pull}196973[#196973]). + +[discrete] +[[enhancements-8.17.0]] +==== Enhancements +* Checks user permissions before initializing the entity engine ({kibana-pull}198661[#198661]). +* Updates LangChain dependencies, adding support for the new Bedrock cross-region inference profiles ({kibana-pull}198622[#198622]). + +[discrete] +[[bug-fixes-8.17.0]] +==== Bug fixes +* Clears the error on the second entity engine initialization ({kibana-pull}202903[#202903]). +* Modifies the empty state message that appears when installing prebuilt rules ({kibana-pull}202226[#202226]). +* Rejects CEF logs from Automatic Import and instead redirects you to the CEF integration ({kibana-pull}201792[#201792], {kibana-pull}202994[#202994]). +* Fixes a bug in Automatic Import where icons did not display after the integration was installed ({kibana-pull}201139[#201139]). +* Removes an erroneous duplicate Preserve Original Event flag as one was additionally added from the common settings file ({kibana-pull}201622[#201622]). +* Turns off the **Install All** button on the **Add Elastic Rules** page while rules are being installed ({kibana-pull}201731[#201731]). +* Turns off the **Add note** button in the alert details flyout if you don't have the appropriate permission ({kibana-pull}201707[#201707]). +* Removes fields with an `@` from the script processor ({kibana-pull}201548[#201548]). +* Fixes an issue that could interfere with Knowledge Base setup ({kibana-pull}201175[#201175]). +* Fixes an issue with Gemini streaming in the AI Assistant ({kibana-pull}201299[#201299]). +* Updates LangChain dependencies, adding support for the new Bedrock cross-region inference endpoints ({kibana-pull}198622[#198622]). +* Fixes a bug with threshold rules that prevented cardinality details from appearing ({kibana-pull}201162[#201162]). +* Fixes a bug that caused an entity engine to get stuck in the `Installing` status if the default Security data view didn't exist. With this fix, engines now correctly report the `Error` state ({kibana-pull}201140[#201140]). +* Fixes an issue that prevented you from successfully importing TSV files with asset criticality data if you're on Windows ({kibana-pull}199791[#199791]). +* Fixes asset criticality index issue when setting up entity engines concurrently ({kibana-pull}199486[#199486]). +* Fixes a bug where the `@timestamp` field wouldn't update upon asset criticality soft delete ({kibana-pull}196722[#196722]). +* Fixes a bug that prevented the save notification from displaying on duplicated Timelines with changes ({kibana-pull}198652[#198652]). +* Improves the flow for the Insights section in the alert details flyout ({kibana-pull}197349[#197349]). +* Fixes an issue where users without the {fleet} `read` permission were blocked from interacting with any onboarding card ({kibana-pull}202413[#202413]). +* Improves {elastic-defend} for Linux endpoints by enabling process information enrichment for file and network events when process events are disabled. +* Improves {elastic-defend} by refactoring the kernel driver to work around a `CRITICAL_PROCESS_DIED` bug check (BSOD) that can occur due to a conflict with CrowdStrike Falcon. +* Fixes an issue in {elastic-defend} versions 8.15.2 and 8.15.3 which can result in Windows boot failure `0xC000007B` referencing `ElasticElam.sys` or recovery mode prompt at boot. We have only received reports of this happening when {elastic-defend} is installed alongside CrowdStrike Falcon. +* Fixes an {elastic-defend} bug where the Linux system call (`setsid`) wasn't properly gathered for RHEL 9/CentOS Stream 9 process events. +* Fixes an issue where {elastic-defend} can enter an infinite loop if an external application opens and retains handles to files within {elastic-defend}s directory while it is processing a `get-file` response action. This can result in {elastic-defend} flooding Elasticsearch with documents until the handles are closed. \ No newline at end of file diff --git a/docs/whats-new.asciidoc b/docs/whats-new.asciidoc index 0b6a37ccb4..ed1bc53f7f 100644 --- a/docs/whats-new.asciidoc +++ b/docs/whats-new.asciidoc @@ -4,172 +4,28 @@ Here are the highlights of what’s new and improved in {elastic-sec}. For detailed information about this release, check out our <>. -Other versions: {security-guide-all}/8.15/whats-new.html[8.15] | {security-guide-all}/8.14/whats-new.html[8.14] | {security-guide-all}/8.13/whats-new.html[8.13] | {security-guide-all}/8.12/whats-new.html[8.12] | {security-guide-all}/8.11/whats-new.html[8.11] | {security-guide-all}/8.10/whats-new.html[8.10] | {security-guide-all}/8.9/whats-new.html[8.9] | {security-guide-all}/8.8/whats-new.html[8.8] | {security-guide-all}/8.7/whats-new.html[8.7] | {security-guide-all}/8.6/whats-new.html[8.6] | {security-guide-all}/8.5/whats-new.html[8.5] | {security-guide-all}/8.4/whats-new.html[8.4] | {security-guide-all}/8.3/whats-new.html[8.3] | {security-guide-all}/8.2/whats-new.html[8.2] | {security-guide-all}/8.1/whats-new.html[8.1] | {security-guide-all}/8.0/whats-new.html[8.0] | {security-guide-all}/7.17/whats-new.html[7.17] | {security-guide-all}/7.16/whats-new.html[7.16] | {security-guide-all}/7.15/whats-new.html[7.15] | {security-guide-all}/7.14/whats-new.html[7.14] | {security-guide-all}/7.13/whats-new.html[7.13] | {security-guide-all}/7.12/whats-new.html[7.12] | {security-guide-all}/7.11/whats-new.html[7.11] | {security-guide-all}/7.10/whats-new.html[7.10] | +Other versions: {security-guide-all}/8.16/whats-new.html[8.16] | {security-guide-all}/8.15/whats-new.html[8.15] | {security-guide-all}/8.14/whats-new.html[8.14] | {security-guide-all}/8.13/whats-new.html[8.13] | {security-guide-all}/8.12/whats-new.html[8.12] | {security-guide-all}/8.11/whats-new.html[8.11] | {security-guide-all}/8.10/whats-new.html[8.10] | {security-guide-all}/8.9/whats-new.html[8.9] | {security-guide-all}/8.8/whats-new.html[8.8] | {security-guide-all}/8.7/whats-new.html[8.7] | {security-guide-all}/8.6/whats-new.html[8.6] | {security-guide-all}/8.5/whats-new.html[8.5] | {security-guide-all}/8.4/whats-new.html[8.4] | {security-guide-all}/8.3/whats-new.html[8.3] | {security-guide-all}/8.2/whats-new.html[8.2] | {security-guide-all}/8.1/whats-new.html[8.1] | {security-guide-all}/8.0/whats-new.html[8.0] | {security-guide-all}/7.17/whats-new.html[7.17] | {security-guide-all}/7.16/whats-new.html[7.16] | {security-guide-all}/7.15/whats-new.html[7.15] | {security-guide-all}/7.14/whats-new.html[7.14] | {security-guide-all}/7.13/whats-new.html[7.13] | {security-guide-all}/7.12/whats-new.html[7.12] | {security-guide-all}/7.11/whats-new.html[7.11] | {security-guide-all}/7.10/whats-new.html[7.10] | {security-guide-all}/7.9/whats-new.html[7.9] // NOTE: The notable-highlights tagged regions are re-used in the Installation and Upgrade Guide. Full URL links are required in tagged regions. // tag::notable-highlights[] [float] -== Generative AI enhancements +== Logsdb index mode with detection rules and alerts -[float] -=== Improved Automatic Import capabilities - -{security-guide}/automatic-import.html[Automatic Import] can now use a larger variety of large language models and accept larger log samples in a wider range of common formats. - -[float] -=== Analyze more alerts with Attack Discovery - -{security-guide}/attack-discovery.html[Attack Discovery] can now analyze up to 500 alerts at once, and provides higher-quality responses. - -[role="screenshot"] -image::whats-new/images/8.16/attck-disc-alerts-number-menu.png[Attack Discovery alert settings,60%] - -[float] -=== Customize Elastic AI Assistant using Knowledge Base - -Elastic AI Assistant's new {security-guide}/ai-assistant-knowledge-base.html[Knowledge Base] feature allows you to specify individual documents or entire indices that AI Assistant will remember and use as context. This improves the relevance, quality, and customization of its responses. - -[role="screenshot"] -image::whats-new/images/8.16/knowledge-base-add-index-config.png[Knowledge Base's Edit index entry menu,80%] - -[float] -== Entity Analytics enhancements - -[float] -=== Manage persisted entity metadata with entity store - -preview:[] The {security-guide}/entity-store.html[entity store] feature allows you to query, reconcile, and maintain entity metadata from various sources, such as ingested logs, integrated identity providers, external asset repositories, and more. By extracting and storing entities from all indices in the {elastic-sec} default data view, the entity store lets you query entity metadata without real-time data searches. - -After you enable the entity store, the Entity Analytics dashboard displays the {security-guide}/detection-entity-dashboard.html#entity-entities[**Entities** section], which offers a comprehensive view of all hosts and users in your environment. You can filter them by their source, entity risk level, and asset criticality level. - -[role="screenshot"] -image::whats-new/images/8.16/entities-section.png[Entities section of the Entity Analytics dashboard] - -[float] -=== Asset criticality is available by default - -The advanced setting for enabling {security-guide}/asset-criticality.html[asset criticality] has been removed, and this feature is now available by default. - -[float] -=== Run entity risk scoring in multiple spaces - -You can now enable and run {security-guide}/entity-risk-scoring.html[entity risk scoring] in multiple {kib} spaces. This allows you to analyze and monitor entity risk in different contexts simultaneously. - -[float] -=== Recalculate entity risk scores after file upload - -When you {security-guide}/asset-criticality.html#bulk-assign-asset-criticality[bulk assign asset criticality] using the file upload feature, the newly assigned criticality levels are automatically factored in during the next hourly risk scoring calculation. You can now manually trigger an immediate recalculation of entity risk scores by clicking **Recalculate entity risk scores now** during the file upload process. - -[role="screenshot"] -image::whats-new/images/8.16/recalc-ers.png[Recalculate entity risk scores] - -[float] -== Detection rules and alerts enhancements - -[float] -=== Enable prebuilt detection rules on installation - -Previously, {security-guide}/prebuilt-rules-management.html#load-prebuilt-rules[installing and enabling prebuilt rules] took two steps. You can now do both in one step with the **Install and enable** option. This works for both single and multiple rules. - -[role="screenshot"] -image::whats-new/images/8.16/install-enable-rules.png[Install and enable rules, 80%] - -[float] -=== Run rules manually - -{security-guide}/rules-ui-management.html#manually-run-rules[Manually run rules] for testing purposes or additional rule coverage. Details about manual runs (such as the status of each run, the total number of runs that will occur, and more) are shown on the **Execution results** tab of the rule details page. - -[role="screenshot"] -image::whats-new/images/8.16/manual-rule-run-table.png[Manual rule run table] - -[float] -=== Exclude cold and frozen data from rules +The {ref}/logs-data-stream.html[logsdb index mode] allows you to store log data more efficiently. If you're considering using it, refer to {security-guide}/detections-logsdb-index-mode-impact.html[Using logsdb index mode with {elastic-sec}] to learn how it can impact your rules and alerts. -Rules that query cold and frozen data tiers might perform more slowly or fail. To ensure that the rules in your {kib} space exclude query results from cold and frozen tiers when executing, configure the `excludedDataTiersForRuleExecution` <>. +NOTE: To use the {ref}/mapping-source-field.html#synthetic-source[synthetic `_source`] feature, you must have the appropriate subscription. Refer to the subscription page for https://www.elastic.co/subscriptions/cloud[{ecloud}] and {subscriptions}[{stack}/self-managed] for the breakdown of available features and their associated subscription tiers. [float] -=== View {es} queries that run during rule execution +== Signature option available for macOS trusted applications conditions -When previewing a rule, you can also {security-guide}/rules-ui-create.html#view-rule-es-queries[learn about its {es} queries], which are submitted when the rule runs. This information can help you identify and troubleshoot potential rule issues. You can also use it to confirm that your rule is retrieving the expected data. This option is provided for {esql} and EQL rules only. +When adding a {security-guide}/trusted-apps-ov.html[trusted application] for macOS, you can now specify conditions based on the application's digital signer—previously only available on Windows. [float] -=== Alert suppression is generally available for more rule types - -{security-guide}/alert-suppression.html[Alert suppression] is generally available for the indicator match, threshold, {ml}, {esql}, and new terms rule types. It is still in technical preview for event correlation rules. - -[float] -== Investigations enhancements - -[float] -=== Add notes to alerts, events, and Timelines - -You can now attach {security-guide}/add-manage-notes.html[notes] to alerts, events, and Timelines, and manage them from the **Notes** page. This provides an easy way to incorporate notes into your investigative workflows to coordinate responses, conduct threat hunting, and share investigative findings. - -[role="screenshot"] -image::whats-new/images/8.16/new-note-alert-event.png[New note added to an alert] - -[float] -=== View analyzed events from the alert details flyout - -preview:[] By enabling the new `securitySolution:enableVisualizationsInFlyout` advanced setting, you can {security-guide}/view-alert-details.html#expanded-visualizations-view[view analyzed alerts and events] in the **Visualize** tab of the alert details flyout. This allows you to maintain the context of the Alerts table during your investigation and provides an easy way to preview related alerts and events. - -[role="screenshot"] -image::whats-new/images/8.16/visualize-tab-lp-alert-details.gif[Examine alert details from event analyzer, 80%] - -[float] -=== Resize alert and event details flyouts - -You can now resize the alert and event details flyouts and choose how they're displayed—over the Alerts table or next to it. - -[role="screenshot"] -image::whats-new/images/8.16/flyout-settings.gif[Change alert details flyout settings] - -[float] -== {elastic-defend} and response actions enhancements - -[float] -=== More SentinelOne third-party response actions - -Additional third-party response actions are available using Elastic's {security-guide}/third-party-actions.html#sentinelone-response-actions[SentinelOne] integration and connector: - -* Get processes -* Terminate a process - -[float] -=== {elastic-defend}'s automated response actions support all rule types - -You can now configure any detection rule type to perform {elastic-defend}'s {security-guide}/automated-response-actions.html[automated response actions]. - -//// -Commenting out until docs are ready - -[float] -=== New rules for {elastic-defend}'s endpoint protection features - -New prebuilt rules tailored for each of {elastic-defend}'s endpoint protection features—malware, ransomware, memory threats, and malicious behavior—allow you to configure actions tailored for detection or prevention of each type. - -[role="screenshot"] -image::whats-new/images/8.16/endpoint-protection-rules.png[Endpoint protection rules] -//// - -[float] -== Cloud Security enhancements - -[float] -=== Ingest third-party cloud security data - -You can now {security-guide}/ingest-third-party-cloud-security-data.html[ingest cloud security data] from several third-party sources—Falco, AWS Security Hub, and Wiz—into {elastic-sec}. The data appears on the **Alerts** and **Findings** pages, and in the user and host details flyouts. - -[role="screenshot"] -image::whats-new/images/8.16/wiz-findings.png[Wiz data on the Findings page] - -[float] -=== Simplify posture data collection with agentless Cloud Security Posture Management deployment - -Elastic's native {security-guide}/cspm.html[Cloud Security Posture Management (CSPM)] integration now supports agentless deployment, giving you an easier and more streamlined way to collect posture data from your cloud service providers. +== Cases action is generally available +The {kibana-ref}/cases-action-type.html[Cases action] feature, first introduced in 8.14, is moving from technical preview to general availability. Use this action to automatically create cases from rules. // end::notable-highlights[]