Skip to content

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
wants to merge 7 commits into
base: main
Choose a base branch
from
Open

(Doc) ILM Force Merge not on HDD and happens on hosting node not current phase tier #130280

wants to merge 7 commits into from
+11 −10

Conversation

stefnestor
Copy link
Contributor

👋 howdy, team!

A handful of changes for the top of the ILM Force Merge doc, please have Dev confirm 🙏

  1. (Main) Elasticsearch Dev now recommends merges target SSD-not-HDD. (Part of the cultural shift of warm tier having HDD to SSD disk, but specifically merges are much faster on SSD.)
  2. Force Merges aren't guaranteed to happen on the phase's data tier ever since Searchable Snapshots have existed. Related info, video. Ex: you can force merge on cold for cold>frozen even though force merge action not allowed in cold & you never force merge on frozen.
  3. added some "how you can accidentally kick this off"

I'm not tied to formatting at all. And kindly fix links as I'm catching up to the new system. However it needs said, kindly help ensure (1) and (2) correct but feel free to revert (3) as desired 🙏 .

@stefnestor
Update ilm-forcemerge.md
3b66fa0 
👋 howdy, team!

A handful of changes
@stefnestor stefnestor added >bug >docs General docs changes Team:Docs Meta label for docs team labels Jun 28, 2025
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jun 28, 2025 •
edited
Loading

🔍 Preview links for changed docs:

🔔 The preview site may take up to 3 minutes to finish building. These links will become live once it completes.

@elasticsearchmachine
Copy link
Collaborator

@stefnestor please enable the option "Allow edits and access to secrets by maintainers" on your PR. For more information, see the documentation.

@elasticsearchmachine
Copy link
Collaborator

Pinging @elastic/es-docs (Team:Docs)

@elasticsearchmachine elasticsearchmachine added v9.2.0 external-contributor Pull request authored by a developer outside the Elasticsearch team labels Jun 28, 2025
@kilfoyle
Copy link
Contributor

kilfoyle commented Jun 30, 2025 •
edited
Loading

Looks good @stefnestor!

I've added a few comments and have tagged the Data Management team about your questions in the PR description.

@kilfoyle kilfoyle requested a review from a team June 30, 2025 18:19
@stefnestor @kilfoyle
grammar 💚
8edeb7f 
Co-authored-by: David Kilfoyle <[email protected]>
@stefnestor @kilfoyle
feedback 🙏
f7165e7 
Co-authored-by: David Kilfoyle <[email protected]>
@stefnestor @kilfoyle
feedback 🙏
194cabb 
Co-authored-by: David Kilfoyle <[email protected]>
kilfoyle
kilfoyle approved these changes Jun 30, 2025
View reviewed changes
Copy link
Contributor

@kilfoyle kilfoyle left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM! 🚀
Thanks Stef!

nielsbauman
nielsbauman reviewed Jul 1, 2025
View reviewed changes
Copy link
Contributor

@nielsbauman nielsbauman left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Hi @stefnestor, thanks for working on this! I left a few first relatively minor comments.

docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md Outdated
::::

Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The node's role frequently matches the data tier of the ILM's phase of the index, you may choose to adjust this behavior.

I'm having trouble understanding this sentence, but maybe it's just me. Could you explain what you mean?

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah @kilfoyle and I tried to has this out but I'm not sure I got it across the line, sorry.

The needed fix from current doc: force merges do not happen on the ilm phase's node tier. They happen on the node hosting the shard, this specifically is different when you use searchable snapshots (e.g. hot>frozen merges on hot inside phase/action/step: frozen/searchable_snapshot/forcemerge).

I've debated just removing you may choose to adjust this behavior. David & I discussed it as relating to disabling ILM Migrate (and using node attribute allocation filters).

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Ah, thanks for the explanation, that helps a lot! How about something like this:

Suggested change
Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example:
Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example:

in combination with your suggestion in the other thread below.
Optionally, we could link to the force_merge_index option on https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-searchable-snapshot#ilm-searchable-snapshot-options, for more details on force merging for searchable snapshots.

Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Sorry, I forgot to mention that we should fix those links that I added. Do you happen to know what the correct links are? I'm not familiar with the new docs system yet.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

David usually fixes them for me as I'm also getting up to speed on the new system 🙏

@stefnestor @nielsbauman
feedback 🙏
878e9f7 
Co-authored-by: Niels Bauman <[email protected]>
@github-actions GitHub Actions
Copy link
Contributor

github-actions bot commented Jul 2, 2025 •
edited
Loading

🔍 Preview links for changed docs

@stefnestor @nielsbauman
feedback 🙏
0ee8c3f 
Co-authored-by: Niels Bauman <[email protected]>
@nielsbauman
Copy link
Contributor

@kilfoyle could you have one final look as well, please? 🙏

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@nielsbauman nielsbauman nielsbauman left review comments

@kilfoyle kilfoyle kilfoyle approved these changes

Assignees
No one assigned
Labels
>bug >docs General docs changes external-contributor Pull request authored by a developer outside the Elasticsearch team Team:Docs Meta label for docs team v9.2.0
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.
4 participants
@stefnestor @elasticsearchmachine @kilfoyle @nielsbauman