Undersized shards
Elastic Cloud Hosted lets you manage and configure one or more deployments of the versioned Elastic Stack, hosted on Elastic Cloud. Availability This functionality is generally available and ready for production usage. Elastic Cloud on Kubernetes extends Kubernetes orchestration capabilities to allow you to deploy and manage components of the Elastic Stack. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Self-managed deployments are Elastic Stack deployments managed without the assistance of an orchestrator. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
ECH
Generally available
ECK
Generally available
ECE
Generally available
Self-managed
Generally available
One or more shards are far below the recommended size range (roughly 10–50 GB). Too many tiny shards increase master metadata load and make searches fan out across more shard copies than the data volume needs.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Index |
| Domains | performance, index-management, shard-allocation |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Minimum optimal shard size in GB | Integer | 1 |
| Exclude indices based on this pattern | List of strings | |
| Minimum indices size to match | Integer | 1 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
Total number of indices with small shards: 12
The index that contains the smallest shard is: metrics-apm-000012, and the shard size is: 200 MB.
See a sample list: logs-prod-000045, logs-prod-000046.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Set up ILM policy
Condition: Always shown for this insight.
For time-based index metrics-apm-000012, create or update its ILM policy so rollovers use larger, better-sized shards.
Merge similar indices
Condition: Shown for non-system indexes.
Merge similar indices to reduce the number of small indices and improve cluster performance and stability.
Reduce shard count
Condition: Shown for non-system indexes.
Use the shrink API to reduce primary shards on logs-prod-000045 from 24 to 12 in logs-prod-000046. See the Shrink API for prerequisites.
POST logs-prod-000045/_shrink/logs-prod-000046?pretty{ "settings": { "index.number_of_shards": 12 } }
Requires the manage cluster privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Index data is partitioned into shards when the index is created. Shard size grows with the data you write into that index.
Shards that stay well below about 10 GB force the cluster to track more shard metadata than the data volume warrants. Searches fan out across more shards, and master-node cluster-state work grows with shard count. Defaults also cap shards per data node (1,000) and per index (1,024).
Keeping shards in a moderate size range helps the cluster recover faster and keeps search more predictable as index count grows.