High replica count on cold nodes

Some indices on cold nodes have a higher replica count than the threshold for their recovery model. Reducing replicas to zero is only safe when the index is fully mounted from a searchable snapshot; other cold indices need a live replica or a fully mounted searchable snapshot to stay recoverable.

Note

For a complete list of insights, refer to AutoOps insights.

Insight details

Field	Value
Component	Elasticsearch
Severity	Low
Scope	Node
Domains	replication, resource-utilization

Example: What you might see in AutoOps

The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.

High replica count on some indices on cold nodes

What was detected

2 replicas on cold nodes for these indices: logs-prod-000045, logs-prod-000046.

Recommendations

Note

AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.

Reduce replicas to zero

Condition: Shown when affected cold index is fully mounted from a searchable snapshot (restored-* naming) and replication factor exceeds the fully mounted cold threshold (0).

Confirm logs-prod-000045 is backed up, then set number_of_replicas to 0 on the cold tier with the action below (currently 2 replicas). Fewer replicas can increase search latency because fewer nodes serve reads.

PUT logs-prod-000045/_settings
{
  "index": {
    "number_of_replicas": 0
  }
}

Note

Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.

Reduce replicas to one

Condition: Shown when affected cold index is not fully mounted from a searchable snapshot and number_of_replicas is greater than 1.

Set number_of_replicas to 1 for logs-prod-000045 (currently 2 replicas) on the cold tier with the action below. Fewer replicas can increase search latency because fewer nodes serve reads.

PUT logs-prod-000045/_settings
{
  "index": {
    "number_of_replicas": 1
  }
}

Note

Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.

Set up fully mounted searchable snapshots

Condition: Shown when affected cold index is not fully mounted from a searchable snapshot.

Move this index to a fully mounted searchable snapshot in your ILM policy for the cold phase. See Searchable snapshot. Fully mounted indices recover from the underlying snapshot, so you can safely run them with zero replicas without losing availability if a node fails.

Background and impact

Each replica is a full copy of a primary shard. Higher replica counts improve availability and search capacity, but they also multiply disk use and write amplification: every index, update, and recovery must reach every replica.

On the cold tier, reducing replicas lowers cost for data that is accessed less often. That tradeoff is safe at zero replicas only when the index is fully mounted from a searchable snapshot, because Elasticsearch can recover the index from the snapshot if a node is lost. If a cold index is not fully mounted and its replicas are removed, losing a node can leave the index unavailable until recovery completes, and searches that need the missing shard can fail.

Guide

Increase tier capacity