From 1005e6bbb5e2a277c54cce06fdedf0d19cdfc806 Mon Sep 17 00:00:00 2001 From: Nastasha Solomon <79124755+nastasha-solomon@users.noreply.github.com> Date: Thu, 19 Oct 2023 13:42:07 -0400 Subject: [PATCH] Document a known issue about detection rules being duplicated upon update (#4022) (cherry picked from commit 254cc025695f517b95cfba1070cb7c80419ea41d) # Conflicts: # docs/release-notes/8.8.asciidoc # docs/release-notes/8.9.asciidoc --- docs/release-notes/8.6.asciidoc | 11 + docs/release-notes/8.7.asciidoc | 3 +- docs/release-notes/8.8.asciidoc | 508 ++++++++++++++++++++++++++++++++ docs/release-notes/8.9.asciidoc | 187 ++++++++++++ 4 files changed, 708 insertions(+), 1 deletion(-) create mode 100644 docs/release-notes/8.8.asciidoc create mode 100644 docs/release-notes/8.9.asciidoc diff --git a/docs/release-notes/8.6.asciidoc b/docs/release-notes/8.6.asciidoc index 0861a96d1e..b2fb9e4e80 100644 --- a/docs/release-notes/8.6.asciidoc +++ b/docs/release-notes/8.6.asciidoc @@ -5,6 +5,11 @@ [[release-notes-8.6.2]] === 8.6.2 +[discrete] +[[known-issue-8.6.2]] +==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. + [discrete] [[bug-fixes-8.6.2]] ==== Bug fixes and enhancements @@ -15,6 +20,11 @@ [[release-notes-8.6.1]] === 8.6.1 +[discrete] +[[known-issue-8.6.1]] +==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. + [discrete] [[bug-fixes-8.6.1]] ==== Bug fixes and enhancements @@ -29,6 +39,7 @@ [discrete] [[known-issue-8.6.0]] ==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. * When using the Osquery Manager integration with {agent}, Osquery results aren't properly written to {es} and, therefore, cannot be viewed in Kibana (https://github.com/elastic/beats/issues/34250)[#34250]). We recommend that Osquery users skip {stack} version 8.6.0 and upgrade to {stack} version 8.6.1 or later when available. * Investigation guides for some prebuilt rules may not render correctly if they include an escaped character (such as `\"`). To resolve this, update your prebuilt rules once you receive a rule update prompt on the Rules page (https://github.com/elastic/detection-rules/pull/2447[#2447]). diff --git a/docs/release-notes/8.7.asciidoc b/docs/release-notes/8.7.asciidoc index 9e4991a72b..d3c7fcf264 100644 --- a/docs/release-notes/8.7.asciidoc +++ b/docs/release-notes/8.7.asciidoc @@ -8,7 +8,7 @@ [discrete] [[known-issue-8.7.1]] ==== Known issues - +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. * Index aliases and some data streams are not properly retrieved by the {elastic-sec} default data view. * The **Add exceptions flyout** loads indefinitely and an out of memory error displays when a rule has a large number of unmapped fields in multiple indices. To avoid this issue, use the <> to manage exceptions. * If you modify an exception item using the <> API and _only_ specify its `item_id`, the exception item is erroneously duplicated. To avoid this issue, you can either: @@ -137,6 +137,7 @@ To find the affected endpoint policy artifacts: [discrete] [[known-issue-8.7.0]] ==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. * After alerts are generated for the first time, you may have to refresh your browser before your alert data appears on pages that use data views (for example, Timeline). Navigating between pages will not work (https://github.com/elastic/security-docs/issues/3046[#3046]). diff --git a/docs/release-notes/8.8.asciidoc b/docs/release-notes/8.8.asciidoc new file mode 100644 index 0000000000..cca1e64a1d --- /dev/null +++ b/docs/release-notes/8.8.asciidoc @@ -0,0 +1,508 @@ +[[release-notes-header-8.8.0]] +== 8.8 + +[discrete] +[[release-notes-8.8.2]] +=== 8.8.2 + +[discrete] +[[known-issue-8.8.2]] +==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. +* Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval. +* Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure. +* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following: + +** Open alerts in Timeline. +** Adjust the Alerts table size. Do one of the following: + +*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. +*** Adjust your OS zoom settings and refresh the page. + +** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.) +* Alerts table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]). ++ +NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents. + ++ +The workaround for this issue depends on the types of alerts you want to display in the Alerts table. Choose the case that's most relevant to you: + ++ +**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields and the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page: + ++ +NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using. ++ + +. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools. +. Go to *Application -> Storage*, then expand *Local Storage*. +. Click on the name of your Kibana instance, for example, http://localhost:1234. +. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alerts table's state, including the table's selected columns. +. Paste the JSON blob into a text file and edit it as follows: +.. Remove the `id:file.name` string from the `columns` array. +.. Remove the `file.name` string from the `visibleColumns` array. +. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key. +. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column. ++ +NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table. + ++ +**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: + +. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. +. Search for the `file.name` field, de-select it, and click *Close*. +. Refresh the Alerts page. + +[discrete] +[[breaking-changes-8.8.2]] +==== Breaking changes + +There are no breaking changes in 8.8.2. + +[discrete] +[[enhancements-8.8.2]] +==== Enhancements +There are no user-facing changes in 8.8.2. + +[discrete] +[[bug-fixes-8.8.2]] +==== Bug fixes +* Fixes a bug that affected links to {kib} results generated by actions on rules from non-default {kib} spaces ({pull}159966[#159966]). +* Fixes a bug that prevented users from saving a Timeline after adding a `number` field ({pull}159723[#159723]). +* Fixes a bug that caused error messages to wrongfully display if users selected uninstalled {ml} jobs while creating a {ml} rule ({pull}159316[#159316]). +* Removes hover actions from tables within the Detection & Response dashboard when the alert count is zero ({pull}158902[#158902]). +* Fixes bugs in the Anomalies table that left {ml} jobs greyed out after they were installed ({pull}158821[#158821]) and that stopped some job counts from appearing ({pull}158739[#158739]). +* Fixes a bug that caused the **Add exceptions flyout** to load indefinitely and display an out of memory error when a rule had a large number of unmapped fields in multiple indices ({pull}159216[#159216]). +* Fixes a bug that prevented cell actions on fields with multiple values in the Alerts table ({pull}158060[#158060]). +* Fixes a bug that caused the Alerts page to query unnecessary indices ({pull}157286[#157286]). +* Fixes a bug that broke the mustache syntax for variables in rule actions ({pull}160446[#160446]). +* Fixes a bug that caused exception items to be erroneously duplicated if you modified an exception item using the <> API and _only_ specified its `item_id` ({pull}159223[#159223]). ++ +NOTE: If you've already encountered this issue and want to find erroneously duplicated exceptions, use the queries provided below. + +** **Query for finding exception documents that were duplicated from only specifying the `item_id`:** ++ +[source,kibana] +---------------------------------- +// Retrieve exception documents grouped by `item_id`. +// Each bucket contains all duplicates of that document. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "match": { + "type": "exception-list" + } + } + ] + } + }, + "aggs": { + "item_id_duplicates": { + "terms": { + "field": "exception-list.item_id", + "min_doc_count": 2 + }, + "aggs": { + "ids": { + "top_hits": { + "size": 100, // Increase this if you may have more duplicates. + "_source": false + } + } + } + } + }, + "size": 0 +} +---------------------------------- + +** **Query for finding exception documents that were duplicated and have lost their `item_id` because their `id` was used to update them:** ++ +[source,kibana] +---------------------------------- +// Each item returned lost its `item_id`, which is expected to be present and unique. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "term": { + "exception-list.list_type": "item" + } + } + ], + "must_not": [ + { + "exists": { + "field": "exception-list.item_id" + } + } + ] + } + } +} +---------------------------------- + + +[discrete] +[[release-notes-8.8.1]] +=== 8.8.1 + +[discrete] +[[known-issue-8.8.1]] +==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. +* Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval. +* Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure. + +* If you modify an exception item using the <> API and _only_ specify its `item_id`, the exception item is erroneously duplicated. To avoid this issue, you can either: + +** <> through the {security-app} UI. +** Specify an exception item's `item_id` _and_ its `id` when modifying an exception through the <> API. + ++ +If you've already encountered this issue and want to find erroneously duplicated exceptions, use the queries provided below. + +** **Query for finding exception documents that were duplicated from only specifying the `item_id`:** ++ +[source,kibana] +---------------------------------- +// Retrieve exception documents grouped by `item_id`. +// Each bucket contains all duplicates of that document. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "match": { + "type": "exception-list" + } + } + ] + } + }, + "aggs": { + "item_id_duplicates": { + "terms": { + "field": "exception-list.item_id", + "min_doc_count": 2 + }, + "aggs": { + "ids": { + "top_hits": { + "size": 100, // Increase this if you may have more duplicates. + "_source": false + } + } + } + } + }, + "size": 0 +} +---------------------------------- + +** **Query for finding exception documents that were duplicated and have lost their `item_id` because their `id` was used to update them:** ++ +[source,kibana] +---------------------------------- +// Each item returned lost its `item_id`, which is expected to be present and unique. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "term": { + "exception-list.list_type": "item" + } + } + ], + "must_not": [ + { + "exists": { + "field": "exception-list.item_id" + } + } + ] + } + } +} +---------------------------------- + +* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following: + +** Open alerts in Timeline. +** Adjust the Alerts table size. Do one of the following: + +*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. +*** Adjust your OS zoom settings and refresh the page. +*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.) + +* Alerts table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]). ++ +NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents. + ++ +The workaround for this issue depends on the types of alerts you want to display in the Alerts table. Choose the case that's most relevant to you: + ++ +**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields and the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page: + ++ +NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using. ++ + +. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools. +. Go to *Application -> Storage*, then expand *Local Storage*. +. Click on the name of your Kibana instance, for example, http://localhost:1234. +. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alerts table's state, including the table's selected columns. +. Paste the JSON blob into a text file and edit it as follows: +.. Remove the `id:file.name` string from the `columns` array. +.. Remove the `file.name` string from the `visibleColumns` array. +. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key. +. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column. ++ +NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table. + ++ +**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: + +. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. +. Search for the `file.name` field, de-select it, and click *Close*. +. Refresh the Alerts page. + +[discrete] +[[breaking-changes-8.8.1]] +==== Breaking changes + +There are no breaking changes in 8.8.1. + +[discrete] +[[features-8.8.1]] +==== New features + +* Introduces the Generative AI connector and <> for {elastic-sec} ({pull}157228[#157228], {pull}156933[#156933]). + +[discrete] +[[bug-fixes-8.8.1]] +==== Bug fixes +* Fixes a bug that made field types appear as `unknown` within the **Fields** browser and when examining alert or event details ({pull}158594[#158594]). +* Fixes a bug that caused all field types in the **Fields** browser to appear as `unknown` ({pull}158594[#158594]). +* Fixes a bug that caused the **Add rule exception** flyout to load indefinitely when index fields couldn't be retrieved ({pull}158371[#158371]). +* Provides support for using field names with wildcards in rule queries ({pull}157981[#157981]). +* Fixes CSS style issues on the rule details page ({pull}157935[#157935]). +* Fixes a bug that caused the `A-Z` option to incorrectly display on Alerts table sorting menus ({pull}157653[#157653]). +* Allows users to scroll through long error messages on the rule details page ({pull}157271[#157271]). + +[discrete] +[[release-notes-8.8.0]] +=== 8.8.0 + +To view a detailed summary of the latest features and enhancements, check out our {security-guide}/whats-new.html[release highlights]. + +[discrete] +[[known-issue-8.8.0]] +==== Known issues +* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. +* Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval. +* Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure. +* {elastic-sec} 8.8 contains a bug that makes field types appear as `unknown` within the **Fields** browser and when examining alert or event details. This bug also causes timestamps to be incorrectly formatted in the Alerts table. To resolve this issue, upgrade to 8.8.1. +* All field types in the **Fields** browser appear as `unknown`. +* If you modify an exception item using the <> API and _only_ specify its `item_id`, the exception item is erroneously duplicated. To avoid this issue, you can either: + +** <> through the {security-app} UI. +** Specify an exception item's `item_id` _and_ its `id` when modifying an exception through the <> API. + ++ +If you've already encountered this issue and want to find erroneously duplicated exceptions, use the queries provided below. + +** **Query for finding exception documents that were duplicated from only specifying the `item_id`:** ++ +[source,kibana] +---------------------------------- +// Retrieve exception documents grouped by `item_id`. +// Each bucket contains all duplicates of that document. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "match": { + "type": "exception-list" + } + } + ] + } + }, + "aggs": { + "item_id_duplicates": { + "terms": { + "field": "exception-list.item_id", + "min_doc_count": 2 + }, + "aggs": { + "ids": { + "top_hits": { + "size": 100, // Increase this if you may have more duplicates. + "_source": false + } + } + } + } + }, + "size": 0 +} +---------------------------------- + +** **Query for finding exception documents that were duplicated and have lost their `item_id` because their `id` was used to update them:** ++ +[source,kibana] +---------------------------------- +// Each item returned lost its `item_id`, which is expected to be present and unique. +GET .kibana*/_search +{ + "query": { + "bool": { + "filter": [ + { + "term": { + "exception-list.list_type": "item" + } + } + ], + "must_not": [ + { + "exists": { + "field": "exception-list.item_id" + } + } + ] + } + } +} +---------------------------------- + +* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following: + +** Open alerts in Timeline. +** Adjust the Alerts table size. Do one of the following: + +*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. +*** Change your OS zoom settings and refresh the page. +*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.) + +* Alerts table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]). ++ +NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents. + ++ +The workaround for this issue depends on the types of alerts you want to display in the Alerts table. Choose the case that's most relevant to you: + ++ +**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields and the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page: + ++ +NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using. ++ + +. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools. +. Go to *Application -> Storage*, then expand *Local Storage*. +. Click on the name of your Kibana instance, for example, http://localhost:1234. +. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alerts table's state, including the table's selected columns. +. Paste the JSON blob into a text file and edit it as follows: +.. Remove the `id:file.name` string from the `columns` array. +.. Remove the `file.name` string from the `visibleColumns` array. +. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key. +. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column. ++ +NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table. + ++ +**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: + +. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. +. Search for the `file.name` field, de-select it, and click *Close*. +. Refresh the Alerts page. + +[discrete] +[[breaking-changes-8.8.0]] +==== Breaking changes + +* The privileges for attaching alerts to cases have changed. Now, you need at least `Read` privileges for Security and `All` privileges for Cases ({pull}147985[#147985]). +* Adds conditional actions to the rules API. In {elastic-sec} 8.7 and earlier, action frequencies were set on a rule level by defining the `throttle` field. In 8.8 and later, action frequencies are set at the action level, and the `throttle` field is replaced by the `frequency` and `alert_filters` fields. The following APIs are affected: +** https://www.elastic.co/guide/en/security/8.8/rules-api-get.html[Get rule] +** https://www.elastic.co/guide/en/security/8.8/rules-api-find.html[Find rules] +** https://www.elastic.co/guide/en/security/8.8/rules-api-create.html#optional-actions-fields-rule-create[Create rule] +** https://www.elastic.co/guide/en/security/8.8/rules-api-update.html#optional-actions-fields-rule-update[Update rule] +** https://www.elastic.co/guide/en/security/8.8/bulk-actions-rules-api.html#optional-actions-fields-bulk-update[Bulk rule actions] + +[discrete] +[[deprecations-8.8.0]] +==== Deprecations + +* The rule level `throttle` field is deprecated in {elastic-sec} 8.8 and is scheduled for end of life in Q4 of 2024. In {elastic-sec} 8.8 and later, we strongly recommend using the action level `frequency` field to set frequencies for individual rule actions. + +[discrete] +[[features-8.8.0]] +==== New features + +* Introduces <>, which scans your cloud VMs for vulnerabilities, and adds a tab to the Findings page that displays vulnerabilities ({pull}154388[#154388], {pull}154873[#154873], {pull}155045[#155045]). +* Introduces <>, which allows you to monitor and protect your Kubernetes workloads. +* Adds a new response action that allows you to execute commands on a selected host ({pull}150202[#150202]). +* Adds the `kibana.alert.url` field to alert documents. This field provides a shareable URL for the alert ({pull}155069[#155069]). +* Adds the ability to duplicate a shared exception list ({pull}154991[#154991]). +* Allows Timeline notes to be deleted ({pull}154834[#154834]). +* Allows you to specify conditions for when rule actions should run ({pull}154680[#154680]). +* Adds the ability to snooze rule notifications from the Rules table, the rule details page, or the Actions tab when editing a rule ({pull}153083[#153083], {pull}155407[#155407], {pull}155612[#155612]). +* Adds controls to the Alerts page that allow you to customize which filters appear at the top of the page ({pull}152450[#152450]). + +[discrete] +[[enhancements-8.8.0]] +==== Enhancements + +* Renames the Notable Anomalies section in the Entity Analytics dashboard to Anomalies ({pull}155687[#155687]). +* Displays additional {ml} anomaly jobs on the Entity Analytics dashboard ({pull}155520[#155520]). +* Makes alert count links on the Entity Analytics dashboard navigate to the Alerts page instead of opening in Timeline ({pull}153372[#153372]). +* Updates the Data Quality dashboard to include a new tree map and storage size metrics for each index ({pull}155581[#155581]). +* Adds cloud infrastructure-related fields to the alert details flyout highlighted fields section ({pull}155247[#155247]). +* Allows you to specify how to handle alert suppression for alerts with missing fields ({pull}155055[#155055]). +* Gives users more control over how they receive alert notifications and lets them define conditions that must be met for a notification to occur ({pull}154526[#154526]). +* Adds a warning message to tell you when a rule has reached the maximum number of alerts limit ({pull}154112[#154112]). +* Updates how browser field descriptions are provided to {kib} ({pull}153498[#153498]). +* Enables multi-level grouping for alerts on the Alerts page, based on various fields ({pull}152862[#152862]). +* Adds links to the Detection & Response and Entity Analytics dashboards that jump to the Alerts page with filters enabled ({pull}152714[#152714]). +* Updates the visualizations throughout {elastic-sec} to Lens visualizations ({pull}150531[#150531]). +* Adds a *Share alert* link to the alert details flyout ({pull}148800[#148800]). +* Adds a warning message to the Rules page when a maintenance window is running ({pull}155386[#155386]). +* Adds a global search bar to the Detections and Response and Entity Analytics dashboards ({pull}156832[#156832]). +* Adds the "Investigate in timeline" inline action to alert counts on the Detections and Response and Entity Analytics dashboards ({pull}154299[#154299]). +* Session view: Makes the row representing the session leader remain visible when you scroll past it, and adds a button to this row that allows you to collapse child processes ({pull}154982[#154982]). +* Reduces Linux process event volume by about 50% by combining `fork`, `exec`, and `end` events when they occur around the same time (does not affect queries of this data) ({pull}153213[#153213]). +* Updates where the technical preview tags appear for host risk score features ({pull}156659[#156659], {pull}156514[#156514]). +* Allows you to use fully qualified domain names (FQDNs) for hosts. To learn how to enable the FQDN feature flag, refer to {fleet-guide}/elastic-agent-standalone-feature-flags.html[Configure feature flags for standalone {agents}]. To learn how to set host names in {fleet}, refer to {fleet-guide}/fleet-settings.html#fleet-agent-hostname-format-settings[Agent Binary Download {fleet} settings]. + +[discrete] +[[bug-fixes-8.8.0]] +==== Bug fixes + +* Fixes a bug that interfered with the default time range when you opened an alert in Timeline ({pull}156884[#156884]). +* Fixes a bug that could cause the Alerts page to become unresponsive after entering an invalid query ({pull}156542[#156542]). +* Updates the colors used for entity analytic graphs to match those used for alert graphs ({pull}156383[#156383]). +* Fixes a bug that caused errors on the Data Quality dashboard when a `basePath` was configured ({pull}156233[#156233]). +* Fixes a bug that could cause problems when different users simultaneously edited a Timeline ({pull}155663[#155663]). +* Fixes a bug that could cause the wrong number of rules to appear in the modal for duplicating rules ({pull}155959[#155959]). +* Fixes a bug that could cause a blank option to appear in the Create rule exception form ({pull}155221[#155221]). +* Fixes issues that affected tags in the Add rule exception component of the Shared Exception Lists page ({pull}155219[#155219]). +* Fixes a bug that displayed an outdated count of affected rules on the Shared Exception Lists page ({pull}155108[#155108]). +* Improves performance for rendering indicator match alerts on the Alerts page ({pull}154821[#154821]). +* Fixes a bug that could affect alert prevalence counts on the Alerts page ({pull}154544[#154544]). +* Fixes a bug that could prevent you from using breadcrumbs to return to the Rules page ({pull}150322[#150322]). +* Fixes a bug that could prevent the *View all open alerts* button on the Detection and Response dashboard from applying the correct filters ({pull}156893[#156893]). +* Fixes several bugs related to session view and and Kubernetes dashboard ({pull}154982[#154982]). +* Fixes the delete index API so it only removes {elastic-sec} 7.x signals indices (`.siem-signals-`), index templates, and ILMs and doesn't delete 8.x alert indices (`.alerts-security.alerts-`). diff --git a/docs/release-notes/8.9.asciidoc b/docs/release-notes/8.9.asciidoc new file mode 100644 index 0000000000..d92e0c37f5 --- /dev/null +++ b/docs/release-notes/8.9.asciidoc @@ -0,0 +1,187 @@ +[[release-notes-header-8.9.0]] +== 8.9 + +[discrete] +[[release-notes-8.9.2]] +=== 8.9.2 + +[discrete] +[[bug-fixes-8.9.2]] +==== Bug fixes + +* Fixes a bug that prevented inline actions on the Alerts page from completing ({pull}165099[#165099]). +* Fixes a bug that prevented blocklist file path entries for Windows and macOS applications from being passed as case insensitive ({pull}164200[#164200]). +* Fixes a bug in the confirmation message that appears when duplicating a single rule ({pull}163908[#163908]). +* Fixes a bug on the rule details page that showed the **Data view** label twice if you were viewing a rule using a data view ({pull}164494[#164494]). +* Fixes a bug that affected Timeline when you investigated an alert created from a rule with exceptions ({pull}162190[#162190]). + +[discrete] +[[release-notes-8.9.1]] +=== 8.9.1 + +[discrete] +[[known-issue-8.9.1]] +==== Known issues + +* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following: + +** Open alerts in Timeline. +** Adjust the Alerts table size. Do one of the following: + +*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. +*** Adjust your OS zoom settings and refresh the page. +*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.) + +* Alert table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]). ++ +NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents. + ++ +The workaround for this issue depends on the types of alerts you want to display. Choose the case that's most relevant to you: + ++ +**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields but the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page: + ++ +NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using. ++ + +. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools. +. Go to *Application -> Storage*, then expand *Local Storage*. +. Click on the name of your Kibana instance, for example, http://localhost:1234. +. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alert table's state, including the table's selected columns. +. Paste the JSON blob into a text file and edit it as follows: +.. Remove the `id:file.name` string from the `columns` array. +.. Remove the `file.name` string from the `visibleColumns` array. +. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key. +. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column. ++ +NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table. + ++ +**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: + +. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. +. Search for the `file.name` field, de-select it, and click *Close*. +. Refresh the Alerts page. + +[discrete] +[[enhancements-8.9.1]] +==== Enhancements +* Event correlation queries and rules can now detect {ref}/eql-syntax.html#eql-missing-events[missing events] in EQL sequences. + +[discrete] +[[bug-fixes-8.9.1]] +==== Bug fixes + +* Fixes a copy to clipboard bug that affected non-ECS fields ({pull}162883[#162883]). +* Fixes number rounding issues in the *Top alerts by* table on the Alerts page ({pull}162647[#162647]). +* Fixes bug that prevented controls from being rendered on {elastic-sec} dashboards ({pull}162514[#162514]). +* Fixes a bug that prevented rule changes from being saved if a rule's action frequency was shorter than the rule run interval ({pull}160798[#160798]). + +[discrete] +[[release-notes-8.9.0]] +=== 8.9.0 + +[discrete] +[[known-issue-8.9.0]] +==== Known issues + +* On the new Detection rule monitoring dashboard, total `Rule executions` will not always equal the sum of `Succeeded`, `Warning`, and `Failed` executions. This is expected because rules can write multiple statuses per execution. One typical example is gap detection: if a rule detects a gap in rule execution it will write an intermediate `Failed` status, then continue to run, and write a final status (such as `Warning`) before finishing its execution. +* Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval. +* The `upload` response action does not report the correct amount of available disk space. The correct amount is approximately four. +* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following: + +** Open alerts in Timeline. +** Adjust the Alerts table size. Do one of the following: + +*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. +*** Adjust your OS zoom settings and refresh the page. +*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.) + +* Alert table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]). ++ +NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents. + ++ +The workaround for this issue depends on the types of alerts you want to display. Choose the case that's most relevant to you: + ++ +**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields but the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page: + ++ +NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using. ++ + +. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools. +. Go to *Application -> Storage*, then expand *Local Storage*. +. Click on the name of your Kibana instance, for example, http://localhost:1234. +. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alert table's state, including the table's selected columns. +. Paste the JSON blob into a text file and edit it as follows: +.. Remove the `id:file.name` string from the `columns` array. +.. Remove the `file.name` string from the `visibleColumns` array. +. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key. +. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column. ++ +NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table. + ++ +**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: + +. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. +. Search for the `file.name` field, de-select it, and click *Close*. +. Refresh the Alerts page. + +[discrete] +[[breaking-changes-8.9.0]] +==== Breaking changes + +There are no breaking changes in 8.9.0. + +[discrete] +[[deprecations-8.9.0]] +==== Deprecations +* Removes the option to use the legacy navigation menu ({pull}158094[#158094]). +* General prebuilt threat indicator match rules were deprecated and replaced with improved indicator-type rules. + +[discrete] +[[features-8.9.0]] +==== New features +* Introduces the `top` command for MacOS and Linux, which shows active processes that {elastic-endpoint} is monitoring and recording. Processes are sorted by how much CPU they are causing {elastic-endpoint} to consume. Learn more about the `top` command by referring to the (https://www.github.com/elastic/endpoint/blob/main/EndpointTopCommand.md[readme]). +* Allows you to install the Cloud Security Posture Management (CSPM) integration via CloudFormation ({pull}159994[#159994]). +* Creates a new dashboard, Cloud Native Vulnerability Management, that provides an overview of vulnerabilities on your cloud hosts ({pull}159699[#159699]). +* Allows you to group vulnerabilities by resource (host) on the Vulnerabilities Findings page, and creates a Resource flyout that displays detailed vulnerability findings for individual hosts ({pull}159873[#159873], {pull}158987[#158987]). +* Adds a new custom dashboard, "Detection rule monitoring" ({pull}159875[#159875]). +* Allows you to anonymize event field values sent to AI Assistant ({pull}159857[#159857]). +* Adds a *Chat* button that opens AI Assistant to the alert details flyout ({pull}159633[#159633]). +* Updates AI Assistant to let you create and delete custom system prompts and default conversations ({pull}159365[#159365]). +* Allows you to add alert tags ({pull}157786[#157786]). +* Adds the ability to automatically isolate a host through a rule’s endpoint response action ({pull}152424[#152424]). +* Moves response actions to General Availability. +* Adds a new response action that allows you to upload files to an endpoint that has {elastic-endpoint} installed ({pull}157208[#157208]). +* Makes the Lateral Movement Detection advanced analytics package General Availability, and adds the ability to detect malicious activities in Windows RDP events (https://github.com/elastic/integrations/pull/6588[#6588]). + +[discrete] +[[enhancements-8.9.0]] +==== Enhancements +* Makes it easier to set up exceptions by auto-populating exception conditions and values with relevant alert data ({pull}159075[#159075]). +* Adds a *Last response* dropdown menu to the Rules table that allows you to filter rules by the status of their last execution ("Succeeded", "Warning", or "Failed") ({pull}159865[#159865]). +* Creates a Lens dashboard for monitoring the use of tokens by AI Assistant ({pull}159075[#159075]). +* Creates a connector for D3 Security ({pull}158569[#158569]). +* Improves the interface for installing and upgrading Elastic prebuilt rules ({pull}158450[#158450]). +* Shows a rule's actions on its details page ({pull}158189[#158189]). +* Allows you to add Lens visualizations to cases from the visualization's *More actions* menu ({pull}154918[#154918]). +* Adds a tooltip to snoozed rules that shows exactly when alerting will resume ({pull}157407[#157407]). +* Enhances the Data Exfiltration Detection package by adding the ability to detect exfiltration anomalies through USB devices and Airdrop (https://github.com/elastic/integrations/pull/6577[#6577]). + +[discrete] +[[bug-fixes-8.9.0]] +==== Bug fixes +* Fixes a bug that caused Elastic prebuilt rules to be erroneously duplicated after you upgraded them ({pull}161331[#161331]). +* Fixes a bug that prevented rule exceptions from being auto-populated when you created a new exception from an alert's **Take action** menu ({pull}159908[#159908]). +* Fixes a UI bug that overlaid **Default Risk score** values as you created a new rule. +* Fixes a bug that restricted the number of cloud accounts that could appear on the Cloud Security Posture dashboard to 10 ({pull}157233[#157233]). +* Fixes a bug that allowed you to save a rule with an alert filter missing a query ({pull}159690[#159690]). +* Fixes unexpected filtering behavior on the Alerts page. Now, when you select a filter that excludes all alerts, an empty table now appears as expected ({pull}160374[#160374]). +* Fixes a UI bug where the **Label** field in the Investigation Guide form incorrectly turns red when the entered value is correct ({pull}160574[#160574], {pull}160577[#160577]). +* Fixes a bug that caused rules to snooze longer than specified ({pull}152873[#152873]).