From 3b66fa0b97feff71c8d21450fbdc1009a224a78e Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Sat, 28 Jun 2025 15:43:12 -0600 Subject: [PATCH 1/7] Update ilm-forcemerge.md MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 👋 howdy, team! A handful of changes --- .../index-lifecycle-actions/ilm-forcemerge.md | 25 ++++++++----------- 1 file changed, 11 insertions(+), 14 deletions(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index 891a619d1bb20..d5b0923272f06 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -7,29 +7,26 @@ mapped_pages: Phases allowed: hot, warm. -[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). - -::::{note} -Shards that are relocating during a `forcemerge` will not be merged. -:::: - +[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is best effort. For example, shards that are relocating during a `forcemerge` will not be merged. To use the `forcemerge` action in the `hot` phase, the `rollover` action **must** be present. If no rollover action is configured, {{ilm-init}} will reject the policy. :::::{admonition} Performance considerations :name: ilm-forcemerge-performance -Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. This can happen when you apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple phases. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated. - -If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). - -::::{important} -This can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. -:::: +Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you +* modify an existing {{ilm-init}} policy's phase `min_age` such that indices trigger into force-merging phase faster. +* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated. +* run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices. +If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). This is considered an expert setting override as this can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. -Force merging will be performed by the nodes within the current phase of the index. A forcemerge in the `hot` phase will use hot nodes with potentially faster nodes, while impacting ingestion more. A forcemerge in the `warm` phase will use warm nodes and potentially take longer to perform, but without impacting ingestion in the `hot` tier. +Force merging will be performed by the node hosting the shard. The [node's role](https://www.elastic.co/docs/deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles#data-node-role) frequently is equal to the [data tier](https://www.elastic.co/docs/manage-data/lifecycle/data-tiers) of the {{ilm-init}}'s phase of the index, but this is not guaranteed. For example, a forcemerge in the +* `hot` phase will use hot nodes; however, performing merges on this potentially higher performance hardware may have a trade off of impacting ingestion. +* `warm` phase will use warm nodes; however, merges may potentially take longer to perform on less performant hardware but will avoid impacting ingestion in the `hot` tier. +* `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-searchable-snapshot) via `force_merge_index` happens on the preceeding data tier. +We recommend that merges be targetted against SSD and not HDD disks. ::::: From 8edeb7f9d214d76d9488fd50adacfe0920152703 Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Mon, 30 Jun 2025 14:48:16 -0600 Subject: [PATCH 2/7] =?UTF-8?q?grammar=20=F0=9F=92=9A?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- .../index-lifecycle-actions/ilm-forcemerge.md | 18 +++++++++++------- 1 file changed, 11 insertions(+), 7 deletions(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index d5b0923272f06..6aeba342051c6 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -7,7 +7,7 @@ mapped_pages: Phases allowed: hot, warm. -[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is best effort. For example, shards that are relocating during a `forcemerge` will not be merged. +[Force merges](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-forcemerge) the index into the specified maximum number of [segments](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-indices-segments). This operation is performed on a best effort basis. For example, shards that are relocating during a `forcemerge` will not be merged. To use the `forcemerge` action in the `hot` phase, the `rollover` action **must** be present. If no rollover action is configured, {{ilm-init}} will reject the policy. @@ -16,15 +16,19 @@ To use the `forcemerge` action in the `hot` phase, the `rollover` action **must* Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you * modify an existing {{ilm-init}} policy's phase `min_age` such that indices trigger into force-merging phase faster. -* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If they meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated. +* apply an {{ilm-init}} policy that includes a force merge action to existing indices. If the indices meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated. * run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices. -If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). This is considered an expert setting override as this can have cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. +If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). -Force merging will be performed by the node hosting the shard. The [node's role](https://www.elastic.co/docs/deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles#data-node-role) frequently is equal to the [data tier](https://www.elastic.co/docs/manage-data/lifecycle/data-tiers) of the {{ilm-init}}'s phase of the index, but this is not guaranteed. For example, a forcemerge in the -* `hot` phase will use hot nodes; however, performing merges on this potentially higher performance hardware may have a trade off of impacting ingestion. -* `warm` phase will use warm nodes; however, merges may potentially take longer to perform on less performant hardware but will avoid impacting ingestion in the `hot` tier. -* `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-searchable-snapshot) via `force_merge_index` happens on the preceeding data tier. +::::{important} +Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. +:::: + +Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-settings.md#node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, but this is not guaranteed. For example: +* A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. +* A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier. +* A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot) using `force_merge_index` happens on the preceeding data tier. We recommend that merges be targetted against SSD and not HDD disks. ::::: From fa317e321bd48b0f8396eed69423689f1fa00e5a Mon Sep 17 00:00:00 2001 From: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> Date: Mon, 30 Jun 2025 17:15:05 -0400 Subject: [PATCH 3/7] Update docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md fix link --- .../elasticsearch/index-lifecycle-actions/ilm-forcemerge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index 6aeba342051c6..ee9daa00bbc8e 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -28,7 +28,7 @@ Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it ca Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-settings.md#node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, but this is not guaranteed. For example: * A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. * A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier. -* A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot) using `force_merge_index` happens on the preceeding data tier. +* A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` happens on the preceeding data tier. We recommend that merges be targetted against SSD and not HDD disks. ::::: From f7165e7855a7d02b9dbca047c7e6aa8b682b2f47 Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Mon, 30 Jun 2025 15:27:29 -0600 Subject: [PATCH 4/7] =?UTF-8?q?feedback=20=F0=9F=99=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- .../elasticsearch/index-lifecycle-actions/ilm-forcemerge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index ee9daa00bbc8e..ee57bba902104 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -25,7 +25,7 @@ If you experience a force merge task queue backlog, you might need to increase t Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. :::: -Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-settings.md#node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, but this is not guaranteed. For example: +Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-settings.md#node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example: * A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. * A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier. * A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` happens on the preceeding data tier. From 194cabbd51eac6ddd8bc5dbc677a5ae45b130605 Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Mon, 30 Jun 2025 15:42:39 -0600 Subject: [PATCH 5/7] =?UTF-8?q?feedback=20=F0=9F=99=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: David Kilfoyle <41695641+kilfoyle@users.noreply.github.com> --- .../elasticsearch/index-lifecycle-actions/ilm-forcemerge.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index ee57bba902104..03abd13f8219e 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -25,7 +25,7 @@ If you experience a force merge task queue backlog, you might need to increase t Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. :::: -Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-settings.md#node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example: +Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example: * A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. * A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier. * A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` happens on the preceeding data tier. From 878e9f78876d7584e59a6f3e53cfdb60da4f93ba Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Tue, 1 Jul 2025 18:44:45 -0600 Subject: [PATCH 6/7] =?UTF-8?q?feedback=20=F0=9F=99=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Niels Bauman <33722607+nielsbauman@users.noreply.github.com> --- .../elasticsearch/index-lifecycle-actions/ilm-forcemerge.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index 03abd13f8219e..3907552e0ede0 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -15,11 +15,11 @@ To use the `forcemerge` action in the `hot` phase, the `rollover` action **must* :name: ilm-forcemerge-performance Force merge is a resource-intensive operation. If too many force merges are triggered at once, it can negatively impact your cluster. For example, this can happen when you -* modify an existing {{ilm-init}} policy's phase `min_age` such that indices trigger into force-merging phase faster. +* modify an existing {{ilm-init}} policy's phase `min_age`, causing indices to trigger the force merge at a faster rate. * apply an {{ilm-init}} policy that includes a force merge action to existing indices. If the indices meet the `min_age` criteria, they can immediately proceed through multiple actions. You can prevent this by increasing the `min_age` or setting `index.lifecycle.origination_date` to change how the index age is calculated. * run the [{{ilm-init}} Move Step API](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-ilm-move-to-step) against multiple indices. -If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). +If you experience a force merge task queue backlog, you might need to increase the size of the force merge threadpool so indices can be force merged in parallel. To do this, configure the `thread_pool.force_merge.size` [cluster setting](https://www.elastic.co/docs/api/doc/elasticsearch/operation/operation-cluster-get-settings). ::::{important} Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. From 0ee8c3fce5ce5867af8db36848134702afe224b9 Mon Sep 17 00:00:00 2001 From: Stef Nestor <26751266+stefnestor@users.noreply.github.com> Date: Wed, 2 Jul 2025 10:24:06 -0600 Subject: [PATCH 7/7] =?UTF-8?q?feedback=20=F0=9F=99=8F?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-authored-by: Niels Bauman <33722607+nielsbauman@users.noreply.github.com> --- .../elasticsearch/index-lifecycle-actions/ilm-forcemerge.md | 4 ++-- 1 file changed, 2 insertions(+), 2 deletions(-) diff --git a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md index 3907552e0ede0..0204478c75c10 100644 --- a/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md +++ b/docs/reference/elasticsearch/index-lifecycle-actions/ilm-forcemerge.md @@ -25,10 +25,10 @@ If you experience a force merge task queue backlog, you might need to increase t Note that `thread_pool.force_merge.size` is an advanced setting. Adjusting it can cause cascading performance impacts. Monitor cluster performance and increment the size of the thread pool slowly to reduce the backlog. :::: -Force merging will be performed by the node hosting the shard. The [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) frequently matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}}'s phase of the index, you may choose to adjust this behavior. For example: +Force merging will be performed by the node hosting the shard. Usually, the [node's role](docs-content://deploy-manage/distributed-architecture/clusters-nodes-shards/node-roles.md#set-node-roles) matches the [data tier](docs-content://manage-data/lifecycle/data-tiers.md) of the {{ilm-init}} phase that the index is in. One of the exceptions is when you have manually disabled [ILM Migrate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-migrate) and have specified custom allocations using [ILM allocate](https://www.elastic.co/docs/reference/elasticsearch/index-lifecycle-actions/ilm-allocate). The other exception is searchable snapshots; force merges for [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` are performed in the phase that the index is in **prior** to the `searchable_snapshot` action. You may want to explicitly choose in which data tier the force merge should occur, for example: * A force merge in the `hot` phase will use hot nodes. Merges may be faster on this potentially higher performance hardware but may have the tradeoff of impacting ingestion. * A force merge in the `warm` phase will use warm nodes. Merges may take longer to perform on potentially lower performance hardware but will avoid impacting ingestion in the `hot` tier. -* A force merge in the `cold` or `frozen` phase [{{ilm-init}} Searchable Snapshots](./ilm-searchable-snapshot.md) using `force_merge_index` happens on the preceeding data tier. +* [{{ilm-init}} Searchable Snapshot](./ilm-searchable-snapshot.md) performance is dependant upon the shard having been force merged, so by default this ILM action will enable `force_merge_index`. This will trigger force merges in the preceding node data tier for `cold` and `frozen` phases. We recommend that merges be targetted against SSD and not HDD disks. :::::