diff --git a/docs/detections/prebuilt-rules-management.asciidoc b/docs/detections/prebuilt-rules-management.asciidoc new file mode 100644 index 0000000000..12a6635633 --- /dev/null +++ b/docs/detections/prebuilt-rules-management.asciidoc @@ -0,0 +1,136 @@ +[[prebuilt-rules-management]] +== Install and manage Elastic prebuilt rules + +:frontmatter-description: Start detections quickly with prebuilt rules designed and updated by Elastic. +:frontmatter-tags-products: [security] +:frontmatter-tags-content-type: [how-to] +:frontmatter-tags-user-goals: [manage] + +Follow these guidelines to start using the {security-app}'s <>, keep them updated, and make sure they have the data needed to run successfully. + +* <> +* <> +* <> +* <> +* <> + +[NOTE] +==== +* Prebuilt rules don't start running by default. You must first install the rules, then enable them. After installation, only a few prebuilt rules will be enabled by default, such as the Endpoint Security rule. + +* You can't modify most settings on Elastic prebuilt rules. You can only edit <> and <>. If you want to modify other settings on a prebuilt rule, you must first duplicate it, then make your changes to the duplicated rule. However, your customized rule is entirely separate from the original prebuilt rule, and will not get updates from Elastic if the prebuilt rule is updated. + +* Automatic updates of Elastic prebuilt rules are supported for the current {elastic-sec} version and the latest three previous minor releases. For example, if you’re on {elastic-sec} 8.10, you’ll be able to use the Rules UI to update your prebuilt rules until {elastic-sec} 8.14 is released. After that point, you can still manually download and install updated prebuilt rules, but you must upgrade to the latest {elastic-sec} version to receive automatic updates. +==== + +[float] +[[load-prebuilt-rules]] +=== Install and enable Elastic prebuilt rules + +. Go to *Rules* -> *Detection rules (SIEM)*. The badge next to *Add Elastic rules* shows the number of prebuilt rules available for installation. ++ +[role="screenshot"] +image::images/prebuilt-rules-add-badge.png[The Add Elastic Rules page] + +. Click *Add Elastic rules*. ++ +TIP: To examine the details of a rule before you install it, select the rule name. This opens the rule details flyout. + +. Do one of the following: +* Install all available rules: Click *Install all*. +* Install a single rule: Click *Install rule* for that rule. +* Install multiple rules: Select the rules and click *Install _x_ selected rule(s)*. ++ +TIP: Use the search bar and *Tags* filter to find the rules you want to install. For example, filter by `OS: Windows` if your environment only includes Windows endpoints. For more on tag categories, refer to <>. ++ +[role="screenshot"] +image::images/prebuilt-rules-add.png[The Add Elastic Rules page] + +. Go back to the *Rules* page, search or filter for any rules you want to run, and do either of the following: + +* Enable a single rule: Turn on the rule's *Enabled* switch. +* Enable multiple rules: Select the rules, then click *Bulk actions* -> *Enable*. + +Once you enable a rule, it starts running on its configured schedule. To confirm that it's running successfully, check its *Last response* status in the rules table, or open the rule's details page and check the <> tab. + +[float] +[[prebuilt-rule-tags]] +=== Prebuilt rule tags + +Each prebuilt rule includes several tags identifying the rule's purpose, detection method, associated resources, and other information to help categorize your rules. These tags are category-value pairs; for example, `OS: Windows` indicates rules designed for Windows endpoints. Categories include: + +* `Data Source`: The application, cloud provider, data shipper, or Elastic integration providing data for the rule. +* `Domain`: A general category of data source types (such as cloud, endpoint, or network). +* `OS`: The host operating system, which could be considered another data source type. +* `Resources`: Additional rule resources such as investigation guides. +* `Rule Type`: Identifies if the rule depends on specialized resources (such as machine learning jobs or threat intelligence indicators), or if it's a higher-order rule built from other rules' alerts. +* `Tactic`: MITRE ATT&CK tactics that the rule addresses. +* `Threat`: Specific threats the rule detects (such as Cobalt Strike or BPFDoor). +* `Use Case`: The type of activity the rule detects and its purpose. Use cases include: +** `Active Directory Monitoring`: Detects changes related to Active Directory. +** `Asset Visibility`: Detects changes to specified asset types. +** `Configuration Audit`: Detects undesirable configuration changes. +** `Guided Onboarding`: Example rule, used for {elastic-sec}'s guided onboarding tour. +** `Identity and Access Audit`: Detects activity related to identity and access management (IAM). +** `Log Auditing`: Detects activity on log configurations or storage. +** `Network Security Monitoring`: Detects network security configuration activity. +** `Threat Detection`: Detects threats. +** `Vulnerability`: Detects exploitation of specific vulnerabilities. + +[float] +[[select-all-prebuilt-rules]] +=== Select and duplicate all prebuilt rules + +. Go to *Rules* -> *Detection rules (SIEM)*, then select the *Elastic rules* filter. +. Click *Select all _x_ rules* above the rules table. +. Click *Bulk actions* -> *Duplicate*. +. Select whether to duplicate the rules' exceptions, then click *Duplicate*. + +You can then modify the duplicated rules and, if required, delete the prebuilt ones. However, your customized rules are entirely separate from the original prebuilt rules, and will not get updates from Elastic if the prebuilt rules are updated. + +[float] +[[update-prebuilt-rules]] +=== Update Elastic prebuilt rules + +Elastic regularly updates prebuilt rules to optimize their performance and ensure they detect the latest threats and techniques. When updated versions are available for your installed prebuilt rules, the *Rule Updates* tab appears on the *Rules* page, allowing you to update your installed rules with the latest versions. + +. Go to *Rules* -> *Detection rules (SIEM)*, then select the *Rule Updates* tab. ++ +NOTE: The *Rule Updates* tab doesn't appear if all your installed prebuilt rules are up to date. ++ +[role="screenshot"] +image::images/prebuilt-rules-update.png[The Rule Updates tab on the Rules page] + +. (Optional) To examine the details of a rule's latest version before you update it, select the rule name. This opens the rule details flyout. ++ +Select the *Updates* tab to view rule changes field by field, or the *JSON view* tab to view changes for the entire rule in JSON format. Both tabs display side-by-side comparisons of the *Current rule* (what you currently have installed) and the *Elastic update* version (what you can choose to install). Deleted characters are highlighted in red; added characters are highlighted in green. ++ +To accept the changes and install the updated version, select *Update*. ++ +[role="screenshot"] +image::images/prebuilt-rules-update-diff.png[Prebuilt rule comparison,75%] + +. Do one of the following to update prebuilt rules on the *Rules* page: +* Update all available rules: Click *Update all*. +* Update a single rule: Click *Update rule* for that rule. +* Update multiple rules: Select the rules and click *Update _x_ selected rule(s)*. ++ +TIP: Use the search bar and *Tags* filter to find the rules you want to update. For example, filter by `OS: Windows` if your environment only includes Windows endpoints. For more on tag categories, refer to <>. + +[float] +[[rule-prerequisites]] +=== Confirm rule prerequisites + +Many Elastic prebuilt rules are designed to work with specific Elastic integrations and data fields. These prerequisites are identified in the *Related integrations* and *Required fields* fields on a rule's details page (*Rules* -> *Detection rules (SIEM)*, then click a rule's name). *Related integrations* also displays each integration's installation status and includes links for installing and configuring the listed integrations. + +Additionally, the *Setup guide* section provides guidance on setting up the rule's requirements. + +[role="screenshot"] +image::images/rule-details-prerequisites.png[Rule details page with Related integrations, Required fields, and Setup guide highlighted] + +You can also check rules' related integrations in the *Installed Rules* and *Rule Monitoring* tables. Click the *integrations* badge to display the related integrations in a popup. + +[role="screenshot"] +image::images/rules-table-related-integrations.png[Rules table with related integrations popup,75%] + +TIP: You can hide the *integrations* badge in the rules tables. Go to *{kib}* -> *Stack Management* -> *Advanced Settings*, then turn off `securitySolution:showRelatedIntegrations`. diff --git a/docs/upgrade/upgrade-security.asciidoc b/docs/upgrade/upgrade-security.asciidoc new file mode 100644 index 0000000000..0b097f486b --- /dev/null +++ b/docs/upgrade/upgrade-security.asciidoc @@ -0,0 +1,181 @@ +[chapter] +[[upgrade-intro]] + += Upgrade {elastic-sec} to {version} + +Before you upgrade {elastic-sec}, take the necessary <>, which will vary depending on what version you are upgrading to: + +* <> +* <> +* <> + +Rolling upgrades are unsupported in {elastic-sec}, which runs within the {kib} application. To upgrade, you must shut down all {kib} instances, install the new software, and restart {kib}. +Upgrading while older {kib} instances are running can cause data loss or upgrade failures. + +[WARNING] +==== +When required, {kib} automatically migrates {kibana-ref}/saved-object-migrations.html[saved objects]. +In case of an upgrade failure, you can roll back to an +earlier version of {kib}. To roll back, you **must** have a +{ref}/snapshot-restore.html[backup snapshot] that includes the `kibana` feature +state. By default, snapshots include the `kibana` feature state. +==== + +[float] +=== Upgrading multiple {kib} instances +When upgrading several {kib} instances connected to the same {es} cluster, +ensure that all outdated instances are shut down before starting the upgrade. + +Rolling upgrades are unsupported in {kib}. However, when outdated instances are shut down, you can start all upgraded instances in parallel, +which allows all instances to participate in the upgrade migration in parallel. + +For large deployments with more than 10 {kib} instances and more than 10,000 saved objects, +you can reduce the upgrade downtime by bringing up a single {kib} instance and waiting for it to +complete the upgrade migration before bringing up the remaining instances. + +IMPORTANT: You can upgrade to pre-release versions for testing, +but upgrading from a pre-release to the Generally Available version is unsupported. +You should use pre-release versions only for testing in a temporary environment. + +[float] +=== Support for Elastic prebuilt detection rule automatic updates +<> are supported for the current {elastic-sec} version and the latest three previous minor releases. For example, if you’re upgrading to {elastic-sec} 8.10, you’ll be able to use the Rules UI to update your prebuilt rules until {elastic-sec} 8.14 is released. After that point, you can still manually download and install updated prebuilt rules, but you must upgrade to the latest {elastic-sec} version to receive automatic updates. + +[float] +[[preventing-migration-failures]] +=== Preparing for migration + +Take these extra steps to ensure you are ready for migration. + +[float] +==== Ensure your {es} cluster is healthy +Problems with your {es} cluster can prevent {kib} upgrades from succeeding. + +During the upgrade process, {kib} creates new indices into which updated documents are written. If a cluster is approaching the low watermark, there's a high risk of {kib} not being able to create these. Reading, transforming, and writing updated documents can be memory intensive, using more available heap than during routine operation. You must ensure that enough heap is available to prevent requests from timing out or throwing errors from circuit breaker exceptions. You should also ensure that all shards are replicated and assigned. + +A healthy cluster has: + + * Enough free disk space, at least twice the amount of storage taken up by the `.kibana` and `.kibana_task_manager` indices + * Sufficient heap size + * A "green" cluster status + +[float] +==== Ensure that all {kib} instances are the same +When you perform an upgrade migration of different {kib} versions, the migration can fail. +Ensure that all {kib} instances are running the same version, configuration, and plugins. + +[float] +==== Back up your data + +Be sure to have a {ref}/snapshot-restore.html[snapshot] of all your data before migrating, so that if something goes wrong during migration, you can restore from the snapshot and try again. + +Review the {kibana-ref}/resolve-migrations-failures.html[common causes of {kib} upgrade failures] and how to prevent them. + + +[float] +[[upgrade-8.x]] +== Upgrade from an earlier 8.x version + +. Review the breaking changes for each product you use and make the necessary changes so your code is compatible with {version}. + +** {apm-guide-ref}/apm-breaking.html[APM {version} breaking changes] +** {beats-ref}/breaking-changes.html[Beats {version} breaking changes] +** {ref}/breaking-changes.html[{es} {version} breaking changes] +** {security-guide}/release-notes.html[{elastic-sec} {version} breaking changes] +** {kibana-ref}/release-notes.html[{kib} {version} breaking changes] +** {logstash-ref}/breaking-changes.html[{ls} {version} breaking changes] ++ +. If you use any {es} plugins, ensure each plugin has a version compatible with {es} version {version}. + +. Test the upgrade in an isolated environment before upgrading your production +cluster. + +. Ensure you have a current snapshot before you start the upgrade. ++ +IMPORTANT: You cannot downgrade {es} nodes after upgrading. +If you cannot complete the upgrade process, +you will need to restore from the snapshot. + +. If you use a separate {ref}/monitoring-production.html[monitoring cluster], you should upgrade the monitoring cluster before the production cluster. Generally, the monitoring cluster and the clusters being monitored should be running the same {stack} version. A monitoring cluster cannot monitor production clusters running newer stack versions. If necessary, the monitoring cluster can monitor production clusters running the latest release of the previous major version. + +[float] +[[upgrade-notes-8.8]] +=== Considerations when upgrading to 8.8 + +After you upgrade to 8.8 or later, frequency settings for <> created in 8.7 or earlier are automatically moved from the rule level to the action level. The action schedules remain the same and will continue to run on their previously specified frequency (`On each rule execution`, `Hourly`, `Daily`, or `Weekly`). + +[float] +[[upgrade-7.17-8x]] +== Upgrade from 7.17 to an 8.x version + +This section provides the necessary steps you need to take before upgrading to 8.x. It also contains new requirements and other pertinent information that affect various features in the {security-app}. + +First, review the breaking changes for each product you use and make the necessary changes so your code is compatible with {version}. + +** {apm-guide-ref}/apm-breaking.html[APM {version} breaking changes] +** {beats-ref}/breaking-changes.html[Beats {version} breaking changes] +** {ref}/breaking-changes.html[{es} {version} breaking changes] +** {security-guide}/release-notes.html[{elastic-sec} {version} breaking changes] +** {kibana-ref}/release-notes.html[{kib} {version} breaking changes] +** {logstash-ref}/breaking-changes.html[{ls} {version} breaking changes] + +[float] +[[reenable-rules-upgrade]] +=== Re-enable disabled rules + +Any active rules when you upgrade from 7.17 to 8.0.1 or newer are automatically disabled, and a tag named `auto_disabled_8.0` is added to those rules for tracking purposes. Once the upgrade is complete, you can filter rules by the newly added tag, then use bulk actions to re-enable them: + +. Go to the Rules page (*Detect -> Rules*). +. From the *Tags* dropdown, search for `auto_disabled_8.0`. +. Click *Select all _x_ rules*, or individually select the rules you want to re-enable. +. Click *Bulk actions -> Enable* to re-enable the rules. + +Alternatively, you can use the <> API to re-enable rules. + +[float] +[[fda-upgrade]] +=== Full Disk Access (FDA) approval for {elastic-endpoint} + +When you manually install {elastic-endpoint}, you must approve a system extension, kernel extension, and enable Full Disk Access (FDA). There is a new FDA requirement in 8.x. Refer to <> to review the required permissions. + +[float] +[[data-views-upgrade]] +=== Requirements to display Data views in the {security-app} + +To make the *Data view* option appear in an environment with legacy alerts, a user with elevated role privileges must visit the {security-app}, open a page that displays alert data (such as the Overview page), then refresh the page. The user's role privileges must allow them to enable the detections feature in a Kibana space. Refer to <> for more information. + +NOTE: If new alerts are generated in an upgraded environment without legacy alerts, refreshing any page with alert data in {elastic-sec} will make the *Data view* option appear in the {elastic-sec} UI. + +[float] +[[alert-schema-upgrade]] +=== New alert schema + +The system index for detection alerts has been renamed from `.siem-signals-` to `.alerts-security.alerts-` and is now a hidden index. Therefore, the schema used for alert documents in {elastic-sec} has changed. Users that access documents in the `.siem-signals` indices using the {elastic-sec} API must modify their API queries and scripts to operate properly on the new 8.x alert documents. Refer to <> and review the new <>. + +[float] +[[preview-upgrade]] +=== New privileges required to view alerts and preview rules + +* To view alerts, users need `manage`, `write`, `read`, and `view_index_metadata` privileges to two new indices, `.alerts-security.alerts` and `.internal.alerts-security.alerts`. Existing users who are upgrading to 8.x can retain their privileges to the `.siem-signals` index. + +* To <>, users need `read` access to the new `.preview.alerts-security.alerts` index. Refer to <> for more information. + +[float] +[[im-rules-upgrade]] +=== Updates to indictor match rules + +Changes to the indicator match rule's <> might require you to update existing rules or create new ones after upgrading to 8.x. Be mindful of the following: + +* If an indicator match rule's default threat indicator path was not defined before the upgrade, it will default to `threatintel.indicator` after the upgrade. This allows the rule to continue using indicator data ingested by {filebeat} version 7.x. If a custom value was defined before the upgrade, the value will not change. +* If an existing indicator match rule was configured to use threat indicator indices generated from {filebeat} version 7.x, updating the default threat indicator path to `threat.indicator` after you upgrade to {stack} version 8.x and {agent} or {filebeat} version 8.x configures the rule to use threat indicator indices generated by those later versions. +* You must create separate rules to query threat intelligence indices created by {filebeat} version 7.x and version 8.x because each version requires a different default threat indicator path value. Review the recommendations for <>. + +[float] +[[upgrade-7.16-8.x]] +== Upgrade from 7.16 or earlier to an 8.x version + +To upgrade from 7.16.0 or earlier to {version}, you must first upgrade your {stack} and {agent}s to 7.17 (refer to {fleet-guide}/upgrade-elastic-agent.html[Upgrade Fleet-managed Elastic Agents]). This enables you to use the {kibana-ref}/upgrade-assistant.html[Upgrade Assistant] to prepare for the upgrade. Before you upgrade, you must resolve all critical issues identified by the Upgrade Assistant. Afterwards, you can <>. + +Initially, {agent}s will be version 7.17. This is fine because {elastic-sec} 8.x supports the last minor release in 7.x (7.17) and any subsequent {elastic-endpoint} versions in 8.x. After the {stack} upgrade, you can decide whether to upgrade {agent}s to 8.x, which is recommended to ensure you get the latest features. + +NOTE: You do not need to shut down your {agent}s or endpoints to upgrade the {stack}. \ No newline at end of file