diff --git a/docs/attack-discovery/attack-discovery.asciidoc b/docs/attack-discovery/attack-discovery.asciidoc index 627ffec344..c8a8ce2177 100644 --- a/docs/attack-discovery/attack-discovery.asciidoc +++ b/docs/attack-discovery/attack-discovery.asciidoc @@ -7,36 +7,38 @@ :frontmatter-tags-content-type: [overview] :frontmatter-tags-user-goals: [get-started] -beta::[] +preview::["This feature is in technical preview. It may change in the future, and you should exercise caution when using it in production environments. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of GA features."] NOTE: This feature is available starting with {elastic-sec} version 8.14.0. -Attack discovery leverages large language models (LLMs) to analyze alerts in your environment and identify threats. Each "discovery" represents a potential attack and describes relationships among multiple alerts to tell you which users and hosts are involved, how alerts correspond to the MITRE ATT&CK matrix, and which threat actor might be responsible. This makes the most of each security analyst's time, helps fight alert fatigue, and can reduce your mean time to respond. - -NOTE: Attack discovery currently only analyzes alerts from the past 24 hours. +Attack discovery leverages large language models (LLMs) to analyze alerts in your environment and identify threats. Each "discovery" represents a potential attack and describes relationships among multiple alerts to tell you which users and hosts are involved, how alerts correspond to the MITRE ATT&CK matrix, and which threat actor might be responsible. This can help make the most of each security analyst's time, fight alert fatigue, and reduce your mean time to respond. This page describes: -* <>. -* <>. -* <>. +* <> +* <> +* <> [[attack-discovery-generate-discoveries]] [discrete] == Generate discoveries -To use Attack discovery: +When you access Attack discovery for the first time, you'll need to select an LLM connector before you can analyze alerts. Attack discovery uses the same LLM connectors as <>. To get started: . Click the **Attack discovery** page from {elastic-sec}'s navigation menu. -. When you open the page for the first time, you'll need to select an LLM connector before you can analyze alerts. Select an existing connector from the dropdown menu, or add a new one. +. Select an existing connector from the dropdown menu, or add a new one. + -NOTE: Attack discovery uses the same LLM connectors as <>. If you've already configured one, you can use it here without further configuration. In general, models with larger context windows are more effective for Attack discovery. +.Recommended models +[sidebar] +-- +While Attack discovery is compatible with many different models, our testing found increased performance with Claude 3 Sonnet and Claude 3 Opus. In general, models with larger context windows are more effective for Attack discovery. +-- + image::images/select-model-empty-state.png[] + . Once you've selected a connector, click **Generate** to start the analysis. -It may take from a few seconds up to several minutes to generate discoveries, depending on the number of alerts and the model you selected. +It may take from a few seconds up to several minutes to generate discoveries, depending on the number of alerts and the model you selected. Note that Attack discovery only analyzes alerts from the past 24 hours. IMPORTANT: Attack discovery uses the same data anonymization settings as <>. To configure which alert fields are sent to the LLM and which of those fields are obfuscated, use the Elastic AI Assistant settings. Consider the privacy policies of third-party LLMs before sending them sensitive data. diff --git a/docs/attack-discovery/images/add-discovery-to-assistant.gif b/docs/attack-discovery/images/add-discovery-to-assistant.gif index 11057cadb1..d2d969d60e 100644 Binary files a/docs/attack-discovery/images/add-discovery-to-assistant.gif and b/docs/attack-discovery/images/add-discovery-to-assistant.gif differ diff --git a/docs/attack-discovery/images/attack-discovery-full-card.png b/docs/attack-discovery/images/attack-discovery-full-card.png index 81b9c1de69..af90d5c604 100644 Binary files a/docs/attack-discovery/images/attack-discovery-full-card.png and b/docs/attack-discovery/images/attack-discovery-full-card.png differ diff --git a/docs/cloud-native-security/session-view.asciidoc b/docs/cloud-native-security/session-view.asciidoc index 7c2084dac0..aa96f64a86 100644 --- a/docs/cloud-native-security/session-view.asciidoc +++ b/docs/cloud-native-security/session-view.asciidoc @@ -33,11 +33,11 @@ but this data is not always collected by default. To confirm that Session View d . Go to *Manage* -> *Policies*, and edit one or more of your {elastic-defend} integration policies. . Select the *Policy settings* tab, then scroll down to the Linux event collection section near the bottom. -. Check the box for *Process* events, and turn on the *Include session data* toggle. +. Check the box for *Process* events, and turn on the *Collect session data* toggle. . If you want to include file and network alerts in Session View, check the boxes for *Network* and *File* events. . If you want to enable terminal output capture, turn on the *Capture terminal output* toggle. -Session View can only display data that was collected by {elastic-defend} when *Include session data* was enabled. When this setting is enabled, {elastic-defend} includes additional process context data in captured process, file, and network events. For more information about the additional +Session View can only display data that was collected by {elastic-defend} when *Collect session data* was enabled. When this setting is enabled, {elastic-defend} includes additional process context data in captured process, file, and network events. For more information about the additional fields collected when this setting is enabled, refer to the https://github.com/elastic/ecs/blob/main/rfcs/text/0030-linux-event-model.md[Linux event model RFC]. @@ -127,7 +127,7 @@ To enable terminal output data capture: . Go to *Manage* -> *Policies*, then select one or more of your {elastic-defend} integration policies to edit. . On the *Policy settings* tab, scroll down to the Linux event collection section near the bottom of the page -and select the *Include session data* and *Capture terminal output* options. +and select the *Collect session data* and *Capture terminal output* options. You can configure several additional settings by clicking *Advanced settings* at the bottom of the page: diff --git a/docs/detections/alerts-view-details.asciidoc b/docs/detections/alerts-view-details.asciidoc index 51192bc970..d0aeca2c37 100644 --- a/docs/detections/alerts-view-details.asciidoc +++ b/docs/detections/alerts-view-details.asciidoc @@ -249,7 +249,7 @@ preview::[] * **Related cases**: Shows cases to which the alert has been added. Click a case's name to open its details. * **Alerts related by source event**: Shows alerts created by the same source event. This can help you find alerts with a shared origin and provide more context about the source event. Click the *Investigate in timeline* button to examine related alerts in Timeline. -* **Alerts related by session**: Shows alerts generated during the same <>. These alerts share the same session ID, which is a unique ID for tracking a given Linux session. To use this feature, you must enable the *Include session data* setting in your {elastic-defend} integration policy. Refer to <> for more information. +* **Alerts related by session**: Shows alerts generated during the same <>. These alerts share the same session ID, which is a unique ID for tracking a given Linux session. To use this feature, you must enable the *Collect session data* setting in your {elastic-defend} integration policy. Refer to <> for more information. * **Alerts related by ancestry**: Shows alerts that are related by process events on the same linear branch. Note that alerts generated from processes on child or related branches are not shown. To further examine alerts, click *Investigate in timeline*. [discrete] diff --git a/docs/getting-started/images/install-endpoint/behavior-protection.png b/docs/getting-started/images/install-endpoint/behavior-protection.png index b8417efd3e..a9fa3d5551 100644 Binary files a/docs/getting-started/images/install-endpoint/behavior-protection.png and b/docs/getting-started/images/install-endpoint/behavior-protection.png differ diff --git a/docs/getting-started/images/install-endpoint/event-collection.png b/docs/getting-started/images/install-endpoint/event-collection.png index 8497821bb0..b408295e7b 100644 Binary files a/docs/getting-started/images/install-endpoint/event-collection.png and b/docs/getting-started/images/install-endpoint/event-collection.png differ diff --git a/docs/getting-started/images/install-endpoint/memory-protection.png b/docs/getting-started/images/install-endpoint/memory-protection.png index 70afdd172e..b5fae9f154 100644 Binary files a/docs/getting-started/images/install-endpoint/memory-protection.png and b/docs/getting-started/images/install-endpoint/memory-protection.png differ diff --git a/docs/getting-started/images/install-endpoint/ransomware-protection.png b/docs/getting-started/images/install-endpoint/ransomware-protection.png index 9bf86788fd..824194f19b 100644 Binary files a/docs/getting-started/images/install-endpoint/ransomware-protection.png and b/docs/getting-started/images/install-endpoint/ransomware-protection.png differ diff --git a/docs/management/admin/blocklist.asciidoc b/docs/management/admin/blocklist.asciidoc index fb7e951761..d303c8f26b 100644 --- a/docs/management/admin/blocklist.asciidoc +++ b/docs/management/admin/blocklist.asciidoc @@ -50,7 +50,7 @@ NOTE: You can also select the `Per Policy` option without immediately assigning . When you're done adding entries to the blocklist, ensure that the blocklist is enabled for the {elastic-defend} integration policies that you just assigned: .. Go to **Manage** -> **Policies**, then click on an integration policy. -.. On the **Policy settings** tab, ensure that the **Malware protections enabled** and **Blocklist enabled** toggles are switched on. Both settings are enabled by default. +.. On the **Policy settings** tab, ensure that the **Malware protections** and **Blocklist** toggles are switched on. Both settings are enabled by default. [discrete] [[manage-blocklist]] diff --git a/docs/serverless/alerts/view-alert-details.mdx b/docs/serverless/alerts/view-alert-details.mdx index c62eef92cd..d4e7c75864 100644 --- a/docs/serverless/alerts/view-alert-details.mdx +++ b/docs/serverless/alerts/view-alert-details.mdx @@ -242,7 +242,7 @@ In the expanded view, corelation data is organized into several tables: * **Suppressed alerts**: Shows how many duplicate alerts were suppressed. This information only appears if alert suppression is enabled for the rule. * **Related cases**: Shows cases to which the alert has been added. Click a case's name to open its details. * **Alerts related by source event**: Shows alerts created by the same source event. This can help you find alerts with a shared origin and provide more context about the source event. Click the **Investigate in timeline** button to examine related alerts in Timeline. -* **Alerts related by session**: Shows alerts generated during the same session. These alerts share the same session ID, which is a unique ID for tracking a given Linux session. To use this feature, you must enable the **Include session data** setting in your ((elastic-defend)) integration policy. Refer to Enable Session View data for more information. +* **Alerts related by session**: Shows alerts generated during the same session. These alerts share the same session ID, which is a unique ID for tracking a given Linux session. To use this feature, you must enable the **Collect session data** setting in your ((elastic-defend)) integration policy. Refer to Enable Session View data for more information. * **Alerts related by ancestry**: Shows alerts that are related by process events on the same linear branch. Note that alerts generated from processes on child or related branches are not shown. To further examine alerts, click **Investigate in timeline**.
diff --git a/docs/serverless/cloud-native-security/session-view.mdx b/docs/serverless/cloud-native-security/session-view.mdx index f5d86078f9..4b30e6168f 100644 --- a/docs/serverless/cloud-native-security/session-view.mdx +++ b/docs/serverless/cloud-native-security/session-view.mdx @@ -37,11 +37,11 @@ but this data is not always collected by default. To confirm that Session View d 1. Go to **Assets** → **Policies**, select a policy and then edit one or more of your ((elastic-defend)) integration policies. 1. Select the **Settings** tab, then scroll down to the Linux event collection section near the bottom. -1. Check the box for **Process** events, and turn on the **Include session data** toggle. +1. Check the box for **Process** events, and turn on the **Collect session data** toggle. 1. If you want to include file and network alerts in Session View, check the boxes for **Network** and **File** events. 1. If you want to enable terminal output capture, turn on the **Capture terminal output** toggle. -Session View can only display data that was collected by ((elastic-defend)) when **Include session data** was enabled. When this setting is enabled, ((elastic-defend)) includes additional process context data in captured process, file, and network events. For more information about the additional +Session View can only display data that was collected by ((elastic-defend)) when **Collect session data** was enabled. When this setting is enabled, ((elastic-defend)) includes additional process context data in captured process, file, and network events. For more information about the additional fields collected when this setting is enabled, refer to the [Linux event model RFC](https://github.com/elastic/ecs/blob/main/rfcs/text/0030-linux-event-model.md).
@@ -127,7 +127,7 @@ To enable terminal output data capture: 1. Go to **Assets** → **Policies**, select a policy and then edit one or more of your ((elastic-defend)) integration policies. 1. On the **Settings** tab, scroll down to the Linux event collection section near the bottom of the page - and select the **Include session data** and **Capture terminal output** options. + and select the **Collect session data** and **Capture terminal output** options. You can configure several additional settings by clicking **Advanced settings** at the bottom of the page: diff --git a/docs/serverless/edr-manage/blocklist.mdx b/docs/serverless/edr-manage/blocklist.mdx index 9d0e497641..24bef6d18f 100644 --- a/docs/serverless/edr-manage/blocklist.mdx +++ b/docs/serverless/edr-manage/blocklist.mdx @@ -66,7 +66,7 @@ By default, a blocklist entry is recognized globally across all hosts running (( 1. When you're done adding entries to the blocklist, ensure that the blocklist is enabled for the ((elastic-defend)) integration policies that you just assigned: 1. Go to **Assets** → **Policies**, then click on an integration policy. - 1. On the **Policy settings** tab, ensure that the **Malware protections enabled** and **Blocklist enabled** toggles are switched on. Both settings are enabled by default. + 1. On the **Policy settings** tab, ensure that the **Malware protections** and **Blocklist** toggles are switched on. Both settings are enabled by default.
diff --git a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-behavior-protection.png b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-behavior-protection.png index b8417efd3e..a9fa3d5551 100644 Binary files a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-behavior-protection.png and b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-behavior-protection.png differ diff --git a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-event-collection.png b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-event-collection.png index 8497821bb0..b408295e7b 100644 Binary files a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-event-collection.png and b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-event-collection.png differ diff --git a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-memory-protection.png b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-memory-protection.png index 70afdd172e..b5fae9f154 100644 Binary files a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-memory-protection.png and b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-memory-protection.png differ diff --git a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-ransomware-protection.png b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-ransomware-protection.png index 9bf86788fd..824194f19b 100644 Binary files a/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-ransomware-protection.png and b/docs/serverless/images/configure-endpoint-integration-policy/-getting-started-install-endpoint-ransomware-protection.png differ diff --git a/docs/upgrade/upgrade-7.17-8.x.asciidoc b/docs/upgrade/upgrade-7.17-8.x.asciidoc new file mode 100644 index 0000000000..38890329f8 --- /dev/null +++ b/docs/upgrade/upgrade-7.17-8.x.asciidoc @@ -0,0 +1,158 @@ +[[upgrade-7.17-8x]] +== Upgrade from 7.17 to an 8.x version + +[float] +=== Why it's important to upgrade + +Upgrading provides you access to {elastic-sec}'s latest features, enhancements, and bug fixes, many of which enable you to save your organization money, respond faster to potential threats, and improve the tools you use to investigate and analyze your data. For more benefits, check out our blog, https://www.elastic.co/blog/top-5-reasons-to-upgrade-elastic-security[Top 5 reasons to upgrade Elastic Security]. Also, it's important to ensure that your deployment is fully maintained and supported. For more information, refer to https://www.elastic.co/support/eol[Elastic's Product End of Life Dates]. + +[float] +=== Plan for your upgrade + +Before upgrading to {elastic-sec} 8.x, consider the following recommendations: + +* Plan for an appropriate amount of time to complete the upgrade. Depending on your configuration and the size of your cluster, the process can take up to a week to complete. + +* Open a https://support.elastic.co[support case] with Elastic to alert our Elastic Support team of your system change. If you need additional assistance, https://www.elastic.co/consulting[Elastic Consulting Services] provides the technical expertise and step-by-step approach for upgrading your Elastic deployment. + +* Choose a version to upgrade to. We recommend the latest minor and patch version. Be sure to upgrade your development or non-production deployment to the same version as your production deployment. + +* Ensure that you have {kibana-ref}/xpack-monitoring.html[stack monitoring] enabled in {kib}. Take note of your current index and search rate. + +* Review your selected version's features, Elastic connectors, integrations, and detection rules to determine if you can replace any customized content with out-of-the-box functionality. This can help reduce your workload and the complexity of your upgrade. + +* If your organization sends alerts (formerly known as signals) to an external SOAR (security orchestration, automation, and response) platform, you may need to change your workflows to accommodate the new <> used in 8.x. + +* Review release notes, deprecations, and breaking changes for {security-guide}/release-notes.html[{elastic-sec}], {ref}/es-release-notes.html[{es}], {kibana-ref}/release-notes.html[{kib}], and, if applicable, {fleet-guide}/release-notes.html[{fleet} and {agent}], {beats-ref}/release-notes.html[{beats}], and {logstash-ref}/releasenotes.html[{ls}]. Identify any issues that might affect your deployment. Work with your Elastic team on any questions you may have. Start with breaking changes for your solution and platform components, such as {es} and {kib}. + +* Schedule a system maintenance window within your organization. + +[float] +=== Pre-upgrade steps + +To prepare for the upgrade process, follow these steps before you start: + +. Do a software version inventory across your entire Elastic deployment, including {es}, {kib}, {agent}, {beats}, and {ls}. + +. If you're running any versions earlier than 7.17, you must first upgrade them all to the latest 7.17.x patch release before upgrading to 8.x. This enables you to use the Upgrade Assistant to upgrade to 8.x. ++ +NOTE: If you're managing your deployments using {ece-ref}/ece-upgrade.html[{ece}] (ECE) or {eck-ref}/k8s-upgrading-eck.html[{eck}] (ECK), you may need to upgrade the system clusters first. + +. Note that alerts in 8.x are written using the `kibana_system` user instead of the `current user`, so any user-modified ingest pipelines configured against the alert index (this is uncommon) will use the `kibana_system` user privileges, which may break your ingest pipeline. If you encounter this issue, please do not update your ingest pipeline. Instead, contact your Elastic Support rep for assistance. + +. Run the {kibana-ref}/upgrade-assistant.html[Upgrade Assistant]. Take note of any critical error messages, then work with your Elastic Support rep to resolve them. ++ +.Requirements +[sidebar] +-- +To run the Upgrade Assistant, you must have the `superuser` role on the cluster. +-- + +. If you're not using {ref}/snapshots-take-snapshot.html#automate-snapshots-slm[{slm} (SLM)], you must set up and configure a policy, then run the policy to create at least one {ref}/snapshot-restore.html[snapshot] -- a backup of indices taken from a running cluster. If you need to roll back during the upgrade process, use a recent snapshot to avoid data loss. Snapshots are {ref}/snapshot-restore.html#how-snapshots-work[incremental] -- depending on the cluster size and the input/output rate, the initial snapshot may take several hours to complete. If you're using SLM, {ref}/snapshots-take-snapshot.html#check-slm-history[check the SLM history] to ensure that snapshots are completing successfully. + +[float] +=== Perform an 8.x upgrade on a deployment + +IMPORTANT: We strongly recommend performing the following steps on a non-production deployment first to address any potential issues before upgrading your production deployments. If you're using a cross-cluster search environment, upgrade your remote deployments first. + +. If you haven't already done so, back up your cluster data to a {ref}/snapshots-take-snapshot.html[snapshot]. + +. We recommend you <> in case there are issues with the detection engine after the upgrade. + +. Upgrade {es}. +** If you're using {ecloud}, we recommend upgrades with no downtime. Refer to {cloud}/ec-upgrade-deployment.html[these instructions]. +** If you're using {ece} (ECE), refer to {ece-ref}/ece-upgrade-deployment.html[these instructions]. +** If you're using {eck} (ECK), refer to {eck-ref}/k8s-upgrading-stack.html[these instructions]. +** If you're upgrading a self-orchestrated deployment, refer to {stack-ref}/upgrading-elasticsearch.html[these instructions] and upgrade the data nodes tier by tier in this order: +... Frozen tier +... Cold tier +... Warm tier +... Hot tier +... Any other nodes not in a tier +... All remaining nodes that are neither master-eligible nor data nodes +... Master-eligible nodes + +. Upgrade {kib}. Refer to {stack-ref}/upgrading-kibana.html[these instructions]. ++ +NOTE: If you're using Elastic Cloud Hosted or {ece}, this is already included in the {es} upgrade. + +. Validate that {es} and {kib} are operating as expected by completing the following checks: +.. For {es}: +... Check the status of your clusters and ensure that they're green by running a `GET _cat/health` API request. For more information, refer to the {ref}/cat-health.html[cat health API documentation]. +... Ensure that the index and search rate are close to what they were before upgrading. Go to **Stack Monitoring** -> **{es}** -> **Overview**. ++ +TIP: You can also check the index document count using the {ref}/cat-indices.html[cat index API]. +... Verify that SLM is taking snapshots by {ref}/snapshots-take-snapshot.html#check-slm-history[checking the SLM history]. +... If you use {ml}, ensure that it is up and running. +.. For {kib}: +... Ensure that you and your users can successfully log in to {kib} and access desired pages. +... Check {kibana-ref}/discover.html[Discover] and verify that the index patterns you typically use are available. +... Verify that your commonly used {kibana-ref}/dashboard.html[dashboards] are available and working properly. +... If you use any Watcher-based {kib} scheduled {kibana-ref}/reporting-getting-started.html[reporting], ensure that it's working properly. + +. Upgrade your ingest components (such as {ls}, {fleet} and {agent}, {beats}, etc.). For details, refer to the {stack-ref}/upgrading-elastic-stack.html[Elastic Stack upgrade docs]. + +. Validate that Ingest is operating correctly. +.. Open *Discover*, go through data views for each of your expected ingest data streams, and ensure that data is being ingested in the expected format and volume. + +. Validate that {elastic-sec} is operating correctly. +.. Re-enable your desired SIEM detection rules (rule management), and ensure that enabled rules are running without errors or warnings (rule monitoring). +.. Ensure that any SOAR workflows that consume alerts are working. +.. Verify that any custom dashboards your team has created are working properly, especially if they operate on alert documents. + +. If you performed these steps on a non-production deployment, repeat these same steps on your production environment. If you're using a cross-cluster search environment and performed these steps on your remote clusters, repeat these same steps on your other deployments. +. Confirm with your appropriate stakeholders that the upgrade process has been successful. + +[float] +=== Post-upgrade steps + +The following sections describe procedures to complete after upgrading {elastic-sec} to 8.x. + +[float] +[[reenable-rules-upgrade]] +==== Re-enable disabled rules + +Any active rules when you upgrade from 7.17 to 8.0.1 or newer are automatically disabled, and a tag named `auto_disabled_8.0` is added to those rules for tracking purposes. Once the upgrade is complete, you can filter rules by the new tag, then use bulk actions to re-enable them: + +. Go to the Rules page (*Detect -> Rules*). +. From the *Tags* dropdown, search for `auto_disabled_8.0`. +. Click *Select all _x_ rules*, or individually select the rules you want to re-enable. +. Click *Bulk actions -> Enable* to re-enable the rules. + +Alternatively, you can use the <> API to re-enable rules. + +[float] +[[fda-upgrade]] +==== Full Disk Access (FDA) approval for {elastic-endpoint} + +When you manually install {elastic-endpoint}, you must approve a system extension, kernel extension, and enable Full Disk Access (FDA). There is a new FDA requirement in 8.x. Refer to <> to review the required permissions. + +[float] +[[data-views-upgrade]] +==== Requirements to display Data views in the {security-app} + +To make the *Data view* option appear in an environment with legacy alerts, a user with elevated role privileges must visit the {security-app}, open a page that displays alert data (at least one alert must be present), and then refresh the page. The user's role privileges must allow them to enable the detections feature in a {kib} space. For more information, refer to <>. + +[float] +[[alert-schema-upgrade]] +==== New alert schema + +The system index for detection alerts has been renamed from `.siem-signals-` to `.alerts-security.alerts-` and is now a hidden index. Therefore, the schema used for alert documents in {elastic-sec} has changed. Users that access documents in the `.siem-signals` indices using the {elastic-sec} API must modify their API queries and scripts to operate properly on the new 8.x alert documents. Refer to <> and review the new <>. + +[float] +[[preview-upgrade]] +==== New privileges required to view alerts and preview rules + +* To view alerts, users need `manage`, `write`, `read`, and `view_index_metadata` privileges for two new indices, `.alerts-security.alerts` and `.internal.alerts-security.alerts`. Existing users who are upgrading to 8.x can retain their privileges to the `.siem-signals` index. + +* To <>, users need `read` access to the new `.preview.alerts-security.alerts` index. Refer to <> for more information. + +[float] +[[im-rules-upgrade]] +==== Updates to indicator match rules + +Changes to the indicator match rule type's <> might require you to update existing rules or create new ones after upgrading to 8.x. Be mindful of the following: + +* If an indicator match rule's default threat indicator path was not defined before the upgrade, it will default to `threatintel.indicator` after the upgrade. This allows the rule to continue using indicator data ingested by {filebeat} version 7.x. If a custom value was defined before the upgrade, the value will not change. +* If an existing indicator match rule was configured to use threat indicator indices generated from {filebeat} version 7.x, updating the default threat indicator path to `threat.indicator` after you upgrade to {stack} version 8.x and {agent} or {filebeat} version 8.x configures the rule to use threat indicator indices generated by those later versions. +* You must create separate rules to query threat intelligence indices created by {filebeat} version 7.x and version 8.x because each version requires a different default threat indicator path value. Review the recommendations for <>. \ No newline at end of file diff --git a/docs/upgrade/upgrade-security.asciidoc b/docs/upgrade/upgrade-security.asciidoc index 0b097f486b..588d5f2ea9 100644 --- a/docs/upgrade/upgrade-security.asciidoc +++ b/docs/upgrade/upgrade-security.asciidoc @@ -1,4 +1,3 @@ -[chapter] [[upgrade-intro]] = Upgrade {elastic-sec} to {version} @@ -104,72 +103,6 @@ you will need to restore from the snapshot. After you upgrade to 8.8 or later, frequency settings for <> created in 8.7 or earlier are automatically moved from the rule level to the action level. The action schedules remain the same and will continue to run on their previously specified frequency (`On each rule execution`, `Hourly`, `Daily`, or `Weekly`). -[float] -[[upgrade-7.17-8x]] -== Upgrade from 7.17 to an 8.x version - -This section provides the necessary steps you need to take before upgrading to 8.x. It also contains new requirements and other pertinent information that affect various features in the {security-app}. - -First, review the breaking changes for each product you use and make the necessary changes so your code is compatible with {version}. - -** {apm-guide-ref}/apm-breaking.html[APM {version} breaking changes] -** {beats-ref}/breaking-changes.html[Beats {version} breaking changes] -** {ref}/breaking-changes.html[{es} {version} breaking changes] -** {security-guide}/release-notes.html[{elastic-sec} {version} breaking changes] -** {kibana-ref}/release-notes.html[{kib} {version} breaking changes] -** {logstash-ref}/breaking-changes.html[{ls} {version} breaking changes] - -[float] -[[reenable-rules-upgrade]] -=== Re-enable disabled rules - -Any active rules when you upgrade from 7.17 to 8.0.1 or newer are automatically disabled, and a tag named `auto_disabled_8.0` is added to those rules for tracking purposes. Once the upgrade is complete, you can filter rules by the newly added tag, then use bulk actions to re-enable them: - -. Go to the Rules page (*Detect -> Rules*). -. From the *Tags* dropdown, search for `auto_disabled_8.0`. -. Click *Select all _x_ rules*, or individually select the rules you want to re-enable. -. Click *Bulk actions -> Enable* to re-enable the rules. - -Alternatively, you can use the <> API to re-enable rules. - -[float] -[[fda-upgrade]] -=== Full Disk Access (FDA) approval for {elastic-endpoint} - -When you manually install {elastic-endpoint}, you must approve a system extension, kernel extension, and enable Full Disk Access (FDA). There is a new FDA requirement in 8.x. Refer to <> to review the required permissions. - -[float] -[[data-views-upgrade]] -=== Requirements to display Data views in the {security-app} - -To make the *Data view* option appear in an environment with legacy alerts, a user with elevated role privileges must visit the {security-app}, open a page that displays alert data (such as the Overview page), then refresh the page. The user's role privileges must allow them to enable the detections feature in a Kibana space. Refer to <> for more information. - -NOTE: If new alerts are generated in an upgraded environment without legacy alerts, refreshing any page with alert data in {elastic-sec} will make the *Data view* option appear in the {elastic-sec} UI. - -[float] -[[alert-schema-upgrade]] -=== New alert schema - -The system index for detection alerts has been renamed from `.siem-signals-` to `.alerts-security.alerts-` and is now a hidden index. Therefore, the schema used for alert documents in {elastic-sec} has changed. Users that access documents in the `.siem-signals` indices using the {elastic-sec} API must modify their API queries and scripts to operate properly on the new 8.x alert documents. Refer to <> and review the new <>. - -[float] -[[preview-upgrade]] -=== New privileges required to view alerts and preview rules - -* To view alerts, users need `manage`, `write`, `read`, and `view_index_metadata` privileges to two new indices, `.alerts-security.alerts` and `.internal.alerts-security.alerts`. Existing users who are upgrading to 8.x can retain their privileges to the `.siem-signals` index. - -* To <>, users need `read` access to the new `.preview.alerts-security.alerts` index. Refer to <> for more information. - -[float] -[[im-rules-upgrade]] -=== Updates to indictor match rules - -Changes to the indicator match rule's <> might require you to update existing rules or create new ones after upgrading to 8.x. Be mindful of the following: - -* If an indicator match rule's default threat indicator path was not defined before the upgrade, it will default to `threatintel.indicator` after the upgrade. This allows the rule to continue using indicator data ingested by {filebeat} version 7.x. If a custom value was defined before the upgrade, the value will not change. -* If an existing indicator match rule was configured to use threat indicator indices generated from {filebeat} version 7.x, updating the default threat indicator path to `threat.indicator` after you upgrade to {stack} version 8.x and {agent} or {filebeat} version 8.x configures the rule to use threat indicator indices generated by those later versions. -* You must create separate rules to query threat intelligence indices created by {filebeat} version 7.x and version 8.x because each version requires a different default threat indicator path value. Review the recommendations for <>. - [float] [[upgrade-7.16-8.x]] == Upgrade from 7.16 or earlier to an 8.x version @@ -178,4 +111,6 @@ To upgrade from 7.16.0 or earlier to {version}, you must first upgrade your {sta Initially, {agent}s will be version 7.17. This is fine because {elastic-sec} 8.x supports the last minor release in 7.x (7.17) and any subsequent {elastic-endpoint} versions in 8.x. After the {stack} upgrade, you can decide whether to upgrade {agent}s to 8.x, which is recommended to ensure you get the latest features. -NOTE: You do not need to shut down your {agent}s or endpoints to upgrade the {stack}. \ No newline at end of file +NOTE: You do not need to shut down your {agent}s or endpoints to upgrade the {stack}. + +include::upgrade-7.17-8.x.asciidoc[] \ No newline at end of file diff --git a/docs/whats-new.asciidoc b/docs/whats-new.asciidoc index 31feeca2e5..9ebc9787d8 100644 --- a/docs/whats-new.asciidoc +++ b/docs/whats-new.asciidoc @@ -4,137 +4,123 @@ Here are the highlights of what’s new and improved in {elastic-sec}. For detailed information about this release, check out our <>. -Other versions: {security-guide-all}/8.12/whats-new.html[8.12] | {security-guide-all}/8.11/whats-new.html[8.11] | {security-guide-all}/8.10/whats-new.html[8.10] | {security-guide-all}/8.9/whats-new.html[8.9] | {security-guide-all}/8.8/whats-new.html[8.8] | {security-guide-all}/8.7/whats-new.html[8.7] | {security-guide-all}/8.6/whats-new.html[8.6] | {security-guide-all}/8.5/whats-new.html[8.5] | {security-guide-all}/8.4/whats-new.html[8.4] | {security-guide-all}/8.3/whats-new.html[8.3] | {security-guide-all}/8.2/whats-new.html[8.2] | {security-guide-all}/8.1/whats-new.html[8.1] | {security-guide-all}/8.0/whats-new.html[8.0] | {security-guide-all}/7.17/whats-new.html[7.17] | {security-guide-all}/7.16/whats-new.html[7.16] | {security-guide-all}/7.15/whats-new.html[7.15] | {security-guide-all}/7.14/whats-new.html[7.14] | {security-guide-all}/7.13/whats-new.html[7.13] | {security-guide-all}/7.12/whats-new.html[7.12] | {security-guide-all}/7.11/whats-new.html[7.11] | {security-guide-all}/7.10/whats-new.html[7.10] | +Other versions: {security-guide-all}/8.13/whats-new.html[8.13] | {security-guide-all}/8.12/whats-new.html[8.12] | {security-guide-all}/8.11/whats-new.html[8.11] | {security-guide-all}/8.10/whats-new.html[8.10] | {security-guide-all}/8.9/whats-new.html[8.9] | {security-guide-all}/8.8/whats-new.html[8.8] | {security-guide-all}/8.7/whats-new.html[8.7] | {security-guide-all}/8.6/whats-new.html[8.6] | {security-guide-all}/8.5/whats-new.html[8.5] | {security-guide-all}/8.4/whats-new.html[8.4] | {security-guide-all}/8.3/whats-new.html[8.3] | {security-guide-all}/8.2/whats-new.html[8.2] | {security-guide-all}/8.1/whats-new.html[8.1] | {security-guide-all}/8.0/whats-new.html[8.0] | {security-guide-all}/7.17/whats-new.html[7.17] | {security-guide-all}/7.16/whats-new.html[7.16] | {security-guide-all}/7.15/whats-new.html[7.15] | {security-guide-all}/7.14/whats-new.html[7.14] | {security-guide-all}/7.13/whats-new.html[7.13] | {security-guide-all}/7.12/whats-new.html[7.12] | {security-guide-all}/7.11/whats-new.html[7.11] | {security-guide-all}/7.10/whats-new.html[7.10] | {security-guide-all}/7.9/whats-new.html[7.9] // NOTE: The notable-highlights tagged regions are re-used in the Installation and Upgrade Guide. Full URL links are required in tagged regions. // tag::notable-highlights[] [float] -== Detection rules and alerts enhancements +== Generative AI enhancements -The following enhancements have been added to detection rules and alerts: [float] -=== Per-field diff for Elastic prebuilt rule updates +=== Attack Discovery -When examining an {security-guide}/prebuilt-rules-management.html#update-prebuilt-rules[updated Elastic prebuilt detection rule], you can now view rule changes field by field as well as in a full JSON view. +{security-guide}/attack-discovery.html[Attack discovery] is a new AI-powered tool that identifies potential attacks and maps connections between alerts to the MITRE ATT&CK® matrix, helping you to fight alert fatigue and reduce your mean time to respond. [role="screenshot"] -image::whats-new/images/8.13/prebuilt-rules-update-diff.png[Prebuilt rule comparison, 85%] +image::whats-new/images/8.14/attack-discovery-full-card.png[Attack discovery detail view] [float] -=== Alert suppression supported for indicator match rules +=== Redesigned Elastic AI Assistant UI -{security-guide}/alert-suppression.html[Alert suppression] now supports the {security-guide}/rules-ui-create.html#create-indicator-rule[indicator match] rule type. You can use it to reduce the number of repeated or duplicate detection alerts created by an indicator match rule. +{security-guide}/security-assistant.html[Elastic AI Assistant] for {elastic-sec} has a redesigned user interface that uses a flyout instead of a popup, aligning it with standard {kib} design patterns. Also, when using OpenAI models, AI Assistant can now "stream" responses, rendering word-by-word rather than appearing as complete text blocks, providing a more conversational experience. [float] -=== Refined header design for alert details flyout +== Entity Analytics enhancements -The header design for the {security-guide}/view-alert-details.html[alert details flyout] has been refined to improve readability and structure. Basic alert details now appear clearer and more organized. - -[role="screenshot"] -image::whats-new/images/8.13/alert-details-flyout-right-panel.png[Right panel of the alert details flyout, 75%] [float] -== Persistence of Data Quality dashboard results +=== Asset criticality file upload -The {security-guide}/data-quality-dash.html[Data Quality dashboard] now retains results across sessions, ensuring continuity of information. Additionally, the dashboard now shows when each index was last checked. +You can {security-guide}/asset-criticality.html#bulk-assign-asset-criticality[bulk assign asset criticality] to multiple entities at a time by importing a text file from your asset management tools. This feature allows you to quickly and easily import a list of entities and their asset criticality levels into the {security-app}. [role="screenshot"] -image::whats-new/images/8.13/data-qual-dash.png[The Data Quality dashboard, 85%] +image::whats-new/images/8.14/asset-criticality-file-upload.gif[Animation of asset criticality file upload,90%] [float] -== Visual event analyzer enhancements +=== Unassign asset criticality -The {security-guide}/visual-event-analyzer.html[Visual event analyzer] UI has been enhanced with the following functionality: +You can unassign {security-guide}/asset-criticality.html[asset criticality] from a host or user if the criticality level is no longer known, or the currently assigned level is incorrect. -* Inline actions and a search bar to the left panel: -+ [role="screenshot"] -image::whats-new/images/8.13/event-details.png[Event details panel, 85%] +image::whats-new/images/8.14/unassign-criticality.png[Unassign asset criticality, 50%] -* A date and time range picker, which allows you to analyze an event within a specific period of time: -+ -[role="screenshot"] -image::whats-new/images/8.13/date-range-selection.png[The date and time range picker, 85%] +[float] +=== Risk scoring engine processes up to 10,000 alerts per entity -* A data view selector, which allows you to filter analyzed events further: -+ -[role="screenshot"] -image::whats-new/images/8.13/data-view-selection.png[The data view selector, 85%] +When calculating {security-guide}/entity-risk-scoring.html[entity risk scores], the risk scoring engine now takes into account a maximum of 10,000 alerts per entity. This ensures that the engine remains operational in environments with extremely large data volume. [float] -== Response actions enhancements +=== Access the entity details flyout from the Entity Analytics dashboard -The following enhancements have been added to response actions: +Clicking on a specific host or user name in the {security-guide}/detection-entity-dashboard.html[Entity Analytics dashboard] now opens the host or user details flyout instead of the host or user details page. This allows you to access entity metadata and risk score information without navigating away from the dashboard. [float] -=== Automated response actions for host processes +=== Entity details flyout shows contribution scores per alert -You can now add {elastic-defend}'s `kill-process` or `suspend-process` {security-guide}/response-actions.html[response actions] to detection rules. This allows you to automatically terminate or suspend a process on an affected host when an event meets the rule's criteria. +The **Risk contributions** section of the {security-guide}/hosts-overview.html#host-details-flyout[entity details flyout] now shows the top 10 alerts that contributed to the latest risk scoring calculation and each alert's contribution score. This makes each entity's risk score easier to understand and gives better insight into which alerts you should investigate at the entity level. [role="screenshot"] -image::whats-new/images/8.13/automated-response-actions.png[Automated response actions, 85%] +image::whats-new/images/8.14/contribution-scores-per-alert.png[Contribution scores for top 10 alerts, 90%] [float] -=== Third-party response actions (SentinelOne) +== Detection rules and alerts enhancements -You can now {security-guide}/third-party-actions.html#sentinelone-response-actions[direct SentinelOne] to perform response actions on protected hosts without leaving the {elastic-sec} UI. You can isolate and release a host from detection alerts and the response console, and view third-party actions in the response actions history log. [float] -== Entity Analytics enhancements +=== Value list improvements -The following enhancements have been added to Entity Analytics: +You can now {security-guide}/value-lists-exceptions.html#edit-value-lists[edit value lists] from the UI, wherever you use them. For example, you can now add items to a value list while creating a rule exception that references that value list. + +[role="screenshot"] +image::whats-new/images/8.14/edit-value-lists.png[Edit items in a value list, 90%] [float] -=== Asset criticality +=== Add ES|QL fields as custom highlighted fields -You can now assign an {security-guide}/asset-criticality.html[asset criticality] level to your entities based on their importance to your organization. For example, you can assign **Extreme impact** to business-critical entities, or **Low impact** to entities that pose minimal risk to your security posture. +When adding custom highlighted fields to an {esql} rule, you can now {security-guide}/rules-ui-create.html#custom-highlighted-esql-fields[specify any fields returned by the rule's query]. This allows you to surface fields that contain useful information for investigating alerts. -The risk scoring engine includes asset criticality as an input when calculating entity risk scores. +[float] +=== Editable setup guide field for detection rules -With asset criticality, you can strengthen your threat detection capabilities by focusing your alert triage, threat-hunting, and investigation activities on high-impact entities. +You can now {security-guide}/rules-ui-create.html#rule-ui-advanced-params[edit the **Setup guide** field] for user-created custom rules. Use this informational field to list rule prerequisites such as required integrations, configuration steps, and anything else needed for the rule to work correctly. [role="screenshot"] -image::whats-new/images/8.13/assign-asset-criticality-host-details.png[Assign asset criticality from the host details page, 85%] +image::whats-new/images/8.14/setup-guide-field.png[Setup guide field] [float] -=== Enhanced host and user details flyouts +=== Alert suppression improvements -The redesigned {security-guide}/hosts-overview.html#host-details-flyout[host details flyout] and {security-guide}/users-page.html#user-details-flyout[user details flyout] allow you to: - -* View entity risk data and all risk contributions. Expand the risk summary section to view details about the entity's risk contributions. -* View and assign asset criticality to your entities. -* View relevant entity details such as the entity ID, when the entity was first and last seen, and the associated IP addresses and operating system. - -[role="screenshot"] -image::whats-new/images/8.13/host-details-flyout.png[Host details flyout, 85%] +In 8.14, we've moved {security-guide}/alert-suppression.html[alert suppression] for custom query rules from technical preview to generally available. We've also added alert suppression to event correlation rules (non-sequence queries only) and new terms rules. [float] -== Cloud Security enhancements +== {elastic-defend} enhancements -The following enhancements have been added to Cloud Security: [float] -=== Benchmark rules can be turned off +=== New malware file scanning options -You can now turn individual {security-guide}/cspm-benchmark-rules.html[benchmark rules] on or off. This allows you to customize your Cloud Security Posture Management (CSPM) and Kubernetes Security Posture Management (KSPM) integrations to reduce noise from benchmark rules that don't apply to your environment. +When configuring {security-guide}/configure-endpoint-integration-policy.html#malware-protection[malware protection], you can choose whether {elastic-defend} scans files when they're modified or executed. This can improve performance on hosts where files are frequently modified, while continuing to identify malware as it attempts to run. [role="screenshot"] -image::whats-new/images/8.13/benchmark-rules.png[Benchmark rules, 85%] +image::whats-new/images/8.14/malware-protection.png[Malware protection section, 80%] [float] -=== Cloud native vulnerability management (CNVM) Findings UI enhancements +=== Automatically register {elastic-defend} as antivirus -The **Vulnerabilities** table on the {security-guide}/vuln-management-findings.html[Findings page] now includes improved grouping capabilities (up to three nested groupings), and more table customization options. +If you're using {elastic-defend}'s malware protection, you can now automatically {security-guide}/configure-endpoint-integration-policy.html#register-as-antivirus[register {elastic-defend} as the antivirus software] for Windows endpoints. -image::whats-new/images/8.13/cnvm-findings-grouped.png[CNVM findings grouped, 85%] +[role="screenshot"] +image::whats-new/images/8.14/register-as-antivirus.png[Register as antivirus section, 80%] [float] -== Custom fields for cases must have a default value +== Cloud Security Posture Management support for AWS GovCloud + +Elastic's {security-guide}/cspm.html[Cloud Security Posture Management (CSPM)] integration now supports AWS GovCloud so you can monitor and track how your GovCloud clusters perform against security benchmarks. + -When adding {security-guide}/cases-open-manage.html#cases-ui-custom-fields[custom fields] to a case, any mandatory fields must have a default value. // end::notable-highlights[] diff --git a/docs/whats-new/images/8.14/asset-criticality-file-upload.gif b/docs/whats-new/images/8.14/asset-criticality-file-upload.gif new file mode 100644 index 0000000000..15a9dafbb2 Binary files /dev/null and b/docs/whats-new/images/8.14/asset-criticality-file-upload.gif differ diff --git a/docs/whats-new/images/8.14/attack-discovery-full-card.png b/docs/whats-new/images/8.14/attack-discovery-full-card.png new file mode 100644 index 0000000000..81b9c1de69 Binary files /dev/null and b/docs/whats-new/images/8.14/attack-discovery-full-card.png differ diff --git a/docs/whats-new/images/8.14/contribution-scores-per-alert.png b/docs/whats-new/images/8.14/contribution-scores-per-alert.png new file mode 100644 index 0000000000..483980f378 Binary files /dev/null and b/docs/whats-new/images/8.14/contribution-scores-per-alert.png differ diff --git a/docs/whats-new/images/8.14/edit-value-lists.png b/docs/whats-new/images/8.14/edit-value-lists.png new file mode 100644 index 0000000000..dd53a8dc11 Binary files /dev/null and b/docs/whats-new/images/8.14/edit-value-lists.png differ diff --git a/docs/whats-new/images/8.14/malware-protection.png b/docs/whats-new/images/8.14/malware-protection.png new file mode 100644 index 0000000000..21f824edec Binary files /dev/null and b/docs/whats-new/images/8.14/malware-protection.png differ diff --git a/docs/whats-new/images/8.14/register-as-antivirus.png b/docs/whats-new/images/8.14/register-as-antivirus.png new file mode 100644 index 0000000000..17e6b9cc5d Binary files /dev/null and b/docs/whats-new/images/8.14/register-as-antivirus.png differ diff --git a/docs/whats-new/images/8.14/setup-guide-field.png b/docs/whats-new/images/8.14/setup-guide-field.png new file mode 100644 index 0000000000..32c298b982 Binary files /dev/null and b/docs/whats-new/images/8.14/setup-guide-field.png differ diff --git a/docs/whats-new/images/8.14/unassign-criticality.png b/docs/whats-new/images/8.14/unassign-criticality.png new file mode 100644 index 0000000000..3c6dcfaa5e Binary files /dev/null and b/docs/whats-new/images/8.14/unassign-criticality.png differ