Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Reword ILM how to guide for revert to ILM policies #4667

Merged
merged 7 commits into from
Dec 12, 2024
Merged

Reword ILM how to guide for revert to ILM policies #4667

Changes from 5 commits
Commits
Show all changes
7 commits
Select commit Hold shift + click to select a range
3612146
Reword ILM how to guide for revert to ILM policies
lahsivjar Dec 11, 2024
91118c1
Fix typo
lahsivjar Dec 11, 2024
3939ced
Remove notes entirely
lahsivjar Dec 11, 2024
2817bb9
Add important section
lahsivjar Dec 11, 2024
ccfb2b4
Reword to be more brief
lahsivjar Dec 11, 2024
d0b7937
Update docs/en/observability/apm/manage-storage/ilm-how-to.asciidoc
lahsivjar Dec 12, 2024
a733087
Update docs/en/observability/apm/manage-storage/ilm-how-to.asciidoc
lahsivjar Dec 12, 2024
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
34 changes: 10 additions & 24 deletions docs/en/observability/apm/manage-storage/ilm-how-to.asciidoc
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -11,27 +11,21 @@
Lifecycle policies allow you to automate the lifecycle of your APM indices as they grow and age.
A default policy is applied to each APM data stream, but can be customized depending on your business needs.

[discrete]
[id="index-lifecycle-policies-default{append-legacy}"]
== Default policies
In the latest version of Elastic APM, clusters are managed by
{ref}/index-lifecycle-management.html[index lifecycle management (ILM)]
to provide the default data retention settings for APM data as well as allow
customization for lifecycle.

[NOTE]
====
// Explain what changed
// Before
Before 8.15, clusters used {ref}/index-lifecycle-management.html[index lifecycle management (ILM)] to provide the default data retention settings for APM data.
// After
As of 8.15, new clusters use {ref}/data-stream-lifecycle.html[data stream lifecycle (DSL)] to provide default data retention settings, but the customization for lifecycle is still performed using ILM policies, which are now configured to override the default DSL polices.

// What this means for existing clusters that are upgraded
For _existing_ clusters upgrading to 8.15, the default ILM policies should still work as expected.
You can can continue using ILM or switch the default to DSL explicitly using the {ref}/data-stream-apis.html[data stream API].

// What this means for new clusters that are created
For _new_ clusters created in 8.15 or later, if you prefer to continue using ILM,
follow the steps in this guide to create a custom ILM policy and add it to the `*@custom` component template for each data stream.
Some older indices might be managed by {ref}/data-stream-lifecycle.html[data stream lifecycle (DSL)] if cluster is upgraded to 8.17.
lahsivjar marked this conversation as resolved.
Show resolved Hide resolved
More details are available in <<apm-release-notes>>.
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Are there going to be more details in the 8.17 release notes? It would be nice to link to something more specific than the general release notes page. I'm kind of surprised there's no release notes page for 8.17 on the main / master branch. 🤔

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@1pkg can you ensure the release notes are on main, 8.x and 8.17 branch please?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@colleenmcginnis Given that we will have the release notes soonish, should we go ahead with merging this first?

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@simitt here's the PR https://github.com/elastic/apm-server/pull/14929/files where I manually copied 8.17 changelog to the main branch. I'm not sure why, but the current minor release automation only creates changelog for specific 8.x and 8.17+ branches.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The PR with the changelog from above was merged.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @1pkg! @lahsivjar I think we should update the release note link to the 8.17 page so it's easier for users to find the relevant information. What do you think?

Suggested change
More details are available in <<apm-release-notes>>.
More details are available in <<apm-release-notes-8.17>>.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think we need this PR, too: elastic/apm-server#14932.

Copy link
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @colleenmcginnis for raising this PR

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

@lahsivjar I'll rebuild this PR after elastic/apm-server#14932 is merged and set this PR to automerge. 😅

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you!

====

[discrete]
[id="index-lifecycle-policies-default{append-legacy}"]
== Default policies

Each APM data stream has its own default lifecycle policy including a delete definition and a rollover definition.

The table below describes the delete definition for each APM data stream.
Expand Down Expand Up @@ -119,14 +113,6 @@ The delete phase permanently removes the index after a time threshold is met.

Rollover (writing to a new index) prevents a single index from growing too large and optimizes indexing and search performance.
Rollover occurs after either an age or size metric is met.
The default rollover definition for each APM data stream is applied based on {ref}/data-stream-lifecycle-settings.html#cluster-lifecycle-default-rollover[`cluster.lifecycle.default.rollover`].

The APM data stream lifecycle policies can be viewed in {kib}:

. To open **Index Management**, find **Stack Management** in the main menu or use the {kibana-ref}/introduction.html#kibana-navigation-search[global search field].
. Select **Component Templates**.
. Search for `apm`.
. Look for templates with `@lifecycle` suffix.

TIP: Default lifecycle policies can change between minor versions. This is not considered a breaking change as index management should continually improve and adapt to new features.

Expand Down