From 43408df4103fe5bb60cd017f3ebd7d7052b5c90e Mon Sep 17 00:00:00 2001 From: Vishal Raj Date: Wed, 4 Sep 2024 15:14:50 +0100 Subject: [PATCH] [apm/known_issues] Add known issues for ILM to DSL switch (#4192) * Add known issues for ILM to DSL switch * Apply suggestions from code review Co-authored-by: Colleen McGinnis * Update ki based on recent changes * update link of fix to the latest one * Update wording for cause * Avoid using abbrev for data stream lifecycle * Apply suggestions from code review Co-authored-by: Colleen McGinnis --------- Co-authored-by: Colleen McGinnis (cherry picked from commit c46cb0e1618686c4cb7e3b1a55482250fe1c6a90) --- .../observability/apm/known-issues.asciidoc | 76 +++++++++++++++++++ 1 file changed, 76 insertions(+) diff --git a/docs/en/observability/apm/known-issues.asciidoc b/docs/en/observability/apm/known-issues.asciidoc index 0b87b2df0f..05fd5e24f3 100644 --- a/docs/en/observability/apm/known-issues.asciidoc +++ b/docs/en/observability/apm/known-issues.asciidoc @@ -19,6 +19,82 @@ If applicable, exact error messages linked to this issue so users searching for Link to fix //// +*Upgrading to v8.15.0 may cause APM indices to lose their lifecycle policy* + +_Elastic Stack versions: 8.15.0_ + +_Fixed in Elastic Stack version 8.15.1_ + +// The conditions in which this issue occurs +The issue only occurs when _upgrading_ the {stack} to 8.15.0. +The issue does _not_ occur when creating a _new_ cluster using 8.15.0. +The issue also does not occur if a custom ILM policy is configured using a custom component template. + +// Describe why it happens +In 8.15.0, APM Server switched to use data stream lifecycle to manage data retention +for APM indices for new deployments as well as for upgraded deployments with default lifecycle +configurations. Unfortunately, since any data stream created before 8.15.0 does not have a data +stream lifecycle configuration, such existing data streams become unmanaged for default +lifecycle configurations. + +// How to fix it +Upgrading to 8.15.1 should fix any new indices created for the data stream. However, +indices created in version 8.15.0 would remain unmanaged if the default ILM policy is +used. One of the following approaches can be adopted to fix the unmanaged indices: +1. Manually delete the indices when they are no longer needed. +2. Explicitly configure APM data streams with the default data stream lifecycle config. +Using this approach would migrate all data streams to use data stream lifecycles, +which should be equivalent to the default ILM policies: + +[source,txt] +---- +PUT _data_stream/traces-apm-*/_lifecycle +{ + "data_retention": "10d" +} + +PUT _data_stream/traces-apm.rum*/_lifecycle +{ + "data_retention": "90d" +} + +PUT _data_stream/traces-apm.sampled*/_lifecycle +{ + "data_retention": "1h" +} + +PUT _data_stream/metrics-apm.*.1m-*/_lifecycle +{ + "data_retention": "90d" +} + +PUT _data_stream/metrics-apm.*.10m-*/_lifecycle +{ + "data_retention": "180d" +} + +PUT _data_stream/metrics-apm.*.60m-*/_lifecycle +{ + "data_retention": "390d" +} + +PUT _data_stream/metrics-apm.internal-*/_lifecycle +{ + "data_retention": "90d" +} + +PUT _data_stream/metrics-apm.app.*/_lifecycle +{ + "data_retention": "90d" +} + +PUT _data_stream/logs-apm.*/_lifecycle +{ + "data_retention": "10d" +} +---- + +// Link to fix if it exists +This issue is fixed in 8.15.1 (https://github.com/elastic/elasticsearch/pull/112432[elastic/elasticsearch#112432]). + [[broken-apm-anomaly-rule]] *Upgrading to v8.13.0 to v8.13.2 breaks APM anomaly rules* + _Elastic Stack versions: 8.13.0, 8.13.1, 8.13.2_ +