Document a known issue about detection rules being duplicated upon up…

…date (#4022)

(cherry picked from commit 254cc02)

# Conflicts:
#	docs/release-notes/8.9.asciidoc

docs/release-notes/8.6.asciidoc

@@ -5,6 +5,11 @@
 [[release-notes-8.6.2]]
 === 8.6.2
 
+[discrete]
+[[known-issue-8.6.2]]
+==== Known issues
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
+
 [discrete]
 [[bug-fixes-8.6.2]]
 ==== Bug fixes and enhancements
@@ -15,6 +20,11 @@
 [[release-notes-8.6.1]]
 === 8.6.1
 
+[discrete]
+[[known-issue-8.6.1]]
+==== Known issues
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
+
 [discrete]
 [[bug-fixes-8.6.1]]
 ==== Bug fixes and enhancements
@@ -29,6 +39,7 @@
 [discrete]
 [[known-issue-8.6.0]]
 ==== Known issues
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * When using the Osquery Manager integration with {agent}, Osquery results aren't properly written to {es} and, therefore, cannot be viewed in Kibana (https://github.com/elastic/beats/issues/34250)[#34250]). We recommend that Osquery users skip {stack} version 8.6.0 and upgrade to {stack} version 8.6.1 or later when available.
 * Investigation guides for some prebuilt rules may not render correctly if they include an escaped character (such as `\"`). To resolve this, update your prebuilt rules once you receive a rule update prompt on the Rules page (https://github.com/elastic/detection-rules/pull/2447[#2447]).
 

docs/release-notes/8.7.asciidoc

@@ -8,7 +8,7 @@
 [discrete]
 [[known-issue-8.7.1]]
 ==== Known issues
-
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * Index aliases and some data streams are not properly retrieved by the {elastic-sec} default data view.   
 * The **Add exceptions flyout** loads indefinitely and an out of memory error displays when a rule has a large number of unmapped fields in multiple indices. To avoid this issue, use the <<exceptions-api-overview,exception APIs>> to manage exceptions.
 * If you modify an exception item using the <<exceptions-api-update-item,update exception item>> API and _only_ specify its `item_id`, the exception item is erroneously duplicated. To avoid this issue, you can either:
@@ -137,6 +137,7 @@ To find the affected endpoint policy artifacts:
 [discrete]
 [[known-issue-8.7.0]]
 ==== Known issues
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * After alerts are generated for the first time, you may have to refresh your browser before your alert data appears on pages that use data views (for example, Timeline). Navigating between pages will not work (https://github.com/elastic/security-docs/issues/3046[#3046]).
 
 

docs/release-notes/8.8.asciidoc

@@ -8,7 +8,7 @@
 [discrete]
 [[known-issue-8.8.2]]
 ==== Known issues
-
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval.
 * Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure.
 * A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following:
@@ -163,7 +163,7 @@ GET .kibana*/_search
 [discrete]
 [[known-issue-8.8.1]]
 ==== Known issues
-
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval.
 * Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure.
 
@@ -316,7 +316,7 @@ To view a detailed summary of the latest features and enhancements, check out ou
 [discrete]
 [[known-issue-8.8.0]]
 ==== Known issues
-
+* After upgrading Elastic prebuilt rules, some rules are erroneously duplicated as custom rules. To remove them, go to the Rules page (**Manage** -> **Rules**), click the **Custom rules** filter next to the search bar, then select and delete the duplicate rules. 
 * Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval.
 * Setting the `max_signals` value higher than the {kibana-ref}/alert-action-settings-kb.html#alert-settings[`xpack.alerting.rules.run.alerts.max`] value will lead to rule failure.
 * {elastic-sec} 8.8 contains a bug that makes field types appear as `unknown` within the **Fields** browser and when examining alert or event details. This bug also causes timestamps to be incorrectly formatted in the Alerts table. To resolve this issue, upgrade to 8.8.1.

docs/release-notes/8.9.asciidoc

@@ -0,0 +1,187 @@
+[[release-notes-header-8.9.0]]
+== 8.9
+
+[discrete]
+[[release-notes-8.9.2]]
+=== 8.9.2
+
+[discrete]
+[[bug-fixes-8.9.2]]
+==== Bug fixes
+
+* Fixes a bug that prevented inline actions on the Alerts page from completing ({pull}165099[#165099]).
+* Fixes a bug that prevented blocklist file path entries for Windows and macOS applications from being passed as case insensitive ({pull}164200[#164200]).
+* Fixes a bug in the confirmation message that appears when duplicating a single rule ({pull}163908[#163908]).
+* Fixes a bug on the rule details page that showed the **Data view** label twice if you were viewing a rule using a data view ({pull}164494[#164494]).
+* Fixes a bug that affected Timeline when you investigated an alert created from a rule with exceptions ({pull}162190[#162190]).
+
+[discrete]
+[[release-notes-8.9.1]]
+=== 8.9.1
+
+[discrete]
+[[known-issue-8.9.1]]
+==== Known issues
+
+* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following:
+
+** Open alerts in Timeline. 
+** Adjust the Alerts table size. Do one of the following:
+
+*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. 
+*** Adjust your OS zoom settings and refresh the page.
+*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.)
+
+* Alert table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]).
++
+NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents.
+
++
+The workaround for this issue depends on the types of alerts you want to display. Choose the case that's most relevant to you:
+
++
+**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields but the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page:
+
++
+NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using.
++
+
+. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools.
+. Go to *Application -> Storage*, then expand *Local Storage*. 
+. Click on the name of your Kibana instance, for example, http://localhost:1234. 
+. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alert table's state, including the table's selected columns. 
+. Paste the JSON blob into a text file and edit it as follows: 
+.. Remove the `id:file.name` string from the `columns` array.  
+.. Remove the `file.name` string from the `visibleColumns` array. 
+. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key.
+. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column.
++
+NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table.   
+
++
+**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: 
+
+. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. 
+. Search for the `file.name` field, de-select it, and click *Close*.
+. Refresh the Alerts page. 
+
+[discrete]
+[[enhancements-8.9.1]]
+==== Enhancements
+* Event correlation queries and rules can now detect {ref}/eql-syntax.html#eql-missing-events[missing events] in EQL sequences.
+
+[discrete]
+[[bug-fixes-8.9.1]]
+==== Bug fixes 
+
+* Fixes a copy to clipboard bug that affected non-ECS fields ({pull}162883[#162883]).
+* Fixes number rounding issues in the *Top alerts by* table on the Alerts page ({pull}162647[#162647]).
+* Fixes bug that prevented controls from being rendered on {elastic-sec} dashboards ({pull}162514[#162514]).
+* Fixes a bug that prevented rule changes from being saved if a rule's action frequency was shorter than the rule run interval ({pull}160798[#160798]).
+
+[discrete]
+[[release-notes-8.9.0]]
+=== 8.9.0
+
+[discrete]
+[[known-issue-8.9.0]]
+==== Known issues
+
+* On the new Detection rule monitoring dashboard, total `Rule executions` will not always equal the sum of `Succeeded`, `Warning`, and `Failed` executions. This is expected because rules can write multiple statuses per execution. One typical example is gap detection: if a rule detects a gap in rule execution it will write an intermediate `Failed` status, then continue to run, and write a final status (such as `Warning`) before finishing its execution.
+* Rule changes can't be saved and existing rule actions are removed if the action's frequency is shorter than the rule's run interval.
+* The `upload` response action does not report the correct amount of available disk space. The correct amount is approximately four.
+* A UI bug can affect the Alerts table height, making it difficult to view alerts. To view alerts, do one of the following:
+
+** Open alerts in Timeline. 
+** Adjust the Alerts table size. Do one of the following:
+
+*** Use the **Fields** browser to remove fields from the Alerts table until the table's width is smaller than its container. 
+*** Adjust your OS zoom settings and refresh the page.
+*** Zoom your browser in and out, then refresh the page. (Row height issues can occur at some zoom levels.)
+
+* Alert table rendering issues occur when threat indicator match alerts contain nested `file.name` values, and the Alerts table displays the `file.name` column. The rendering issues stem from a known bug that occurs when the Alerts table sends a request to the Elasticsearch Fields API (https://github.com/elastic/elasticsearch/issues/97684[#97684]).
++
+NOTE: When customizing the Alerts table, avoid adding or displaying fields that also exist as nested properties within any alert documents.
+
++
+The workaround for this issue depends on the types of alerts you want to display. Choose the case that's most relevant to you:
+
++
+**Case #1:** You want to display threat indicator match alerts with nested `file.name` fields but the Alerts table won't render. To fix this, manually edit your browser's local storage and refresh the Alerts page:
+
++
+NOTE: These instructions only apply to the Google Chrome browser. Modify the steps based on of browser you're using.
++
+
+. Right-click anywhere on the Alerts page, then select *Inspect* to open Chrome's Developer Tools.
+. Go to *Application -> Storage*, then expand *Local Storage*. 
+. Click on the name of your Kibana instance, for example, http://localhost:1234. 
+. Search for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key and copy its value. The value you copied is the JSON blob that's used to persist the Alert table's state, including the table's selected columns. 
+. Paste the JSON blob into a text file and edit it as follows: 
+.. Remove the `id:file.name` string from the `columns` array.  
+.. Remove the `file.name` string from the `visibleColumns` array. 
+. Go back to Chrome's Developer Tools, and paste the edited JSON into the value for the `detection-engine-alert-table-securitySolution-rule-details-gridView` key.
+. Click the *Enter* or *Return* key on your keyboard, and refresh the Alerts page. The Alerts table re-renders without the `file.name` column.
++
+NOTE: To avoid further issues, _do not_ re-add the `file.name` field to the table.   
+
++
+**Case #2:** You want to display threat indicator match alerts with nested `file.name` fields and other types of alerts, but the Alerts table is rendering with empty rows. To resolve this issue: 
+
+. Go to the toolbar in the upper-left of the Alerts table, and click *Fields*. 
+. Search for the `file.name` field, de-select it, and click *Close*.
+. Refresh the Alerts page. 
+
+[discrete]
+[[breaking-changes-8.9.0]]
+==== Breaking changes
+
+There are no breaking changes in 8.9.0.
+
+[discrete]
+[[deprecations-8.9.0]]
+==== Deprecations
+* Removes the option to use the legacy navigation menu ({pull}158094[#158094]).
+* General prebuilt threat indicator match rules were deprecated and replaced with improved indicator-type rules. 
+
+[discrete]
+[[features-8.9.0]]
+==== New features
+* Introduces the `top` command for MacOS and Linux, which shows active processes that {elastic-endpoint} is monitoring and recording. Processes are sorted by how much CPU they are causing {elastic-endpoint} to consume. Learn more about the `top` command by referring to the (https://www.github.com/elastic/endpoint/blob/main/EndpointTopCommand.md[readme]).
+* Allows you to install the Cloud Security Posture Management (CSPM) integration via CloudFormation ({pull}159994[#159994]).
+* Creates a new dashboard, Cloud Native Vulnerability Management, that provides an overview of vulnerabilities on your cloud hosts ({pull}159699[#159699]).
+* Allows you to group vulnerabilities by resource (host) on the Vulnerabilities Findings page, and creates a Resource flyout that displays detailed vulnerability findings for individual hosts ({pull}159873[#159873], {pull}158987[#158987]).
+* Adds a new custom dashboard, "Detection rule monitoring" ({pull}159875[#159875]).
+* Allows you to anonymize event field values sent to AI Assistant ({pull}159857[#159857]).
+* Adds a *Chat* button that opens AI Assistant to the alert details flyout ({pull}159633[#159633]).
+* Updates AI Assistant to let you create and delete custom system prompts and default conversations ({pull}159365[#159365]).
+* Allows you to add alert tags ({pull}157786[#157786]).
+* Adds the ability to automatically isolate a host through a rule’s endpoint response action ({pull}152424[#152424]). 
+* Moves response actions to General Availability.
+* Adds a new response action that allows you to upload files to an endpoint that has {elastic-endpoint} installed ({pull}157208[#157208]).
+* Makes the Lateral Movement Detection advanced analytics package General Availability, and adds the ability to detect malicious activities in Windows RDP events (https://github.com/elastic/integrations/pull/6588[#6588]).
+
+[discrete]
+[[enhancements-8.9.0]]
+==== Enhancements
+* Makes it easier to set up exceptions by auto-populating exception conditions and values with relevant alert data  ({pull}159075[#159075]).  
+* Adds a *Last response* dropdown menu to the Rules table that allows you to filter rules by the status of their last execution ("Succeeded", "Warning", or "Failed") ({pull}159865[#159865]).
+* Creates a Lens dashboard for monitoring the use of tokens by AI Assistant ({pull}159075[#159075]).
+* Creates a connector for D3 Security ({pull}158569[#158569]).
+* Improves the interface for installing and upgrading Elastic prebuilt rules ({pull}158450[#158450]).
+* Shows a rule's actions on its details page ({pull}158189[#158189]).
+* Allows you to add Lens visualizations to cases from the visualization's *More actions* menu ({pull}154918[#154918]).
+* Adds a tooltip to snoozed rules that shows exactly when alerting will resume ({pull}157407[#157407]).
+* Enhances the Data Exfiltration Detection package by adding the ability to detect exfiltration anomalies through USB devices and Airdrop (https://github.com/elastic/integrations/pull/6577[#6577]).
+
+[discrete]
+[[bug-fixes-8.9.0]]
+==== Bug fixes
+* Fixes a bug that caused Elastic prebuilt rules to be erroneously duplicated after you upgraded them ({pull}161331[#161331]).   
+* Fixes a bug that prevented rule exceptions from being auto-populated when you created a new exception from an alert's **Take action** menu ({pull}159908[#159908]). 
+* Fixes a UI bug that overlaid **Default Risk score** values as you created a new rule.
+* Fixes a bug that restricted the number of cloud accounts that could appear on the Cloud Security Posture dashboard to 10 ({pull}157233[#157233]).
+* Fixes a bug that allowed you to save a rule with an alert filter missing a query ({pull}159690[#159690]).
+* Fixes unexpected filtering behavior on the Alerts page. Now, when you select a filter that excludes all alerts, an empty table now appears as expected ({pull}160374[#160374]).
+* Fixes a UI bug where the **Label** field in the Investigation Guide form incorrectly turns red when the entered value is correct ({pull}160574[#160574], {pull}160577[#160577]).
+* Fixes a bug that caused rules to snooze longer than specified ({pull}152873[#152873]).