Increase in shards count
Elastic Cloud Hosted lets you manage and configure one or more deployments of the versioned Elastic Stack, hosted on Elastic Cloud. Availability This functionality is generally available and ready for production usage. Elastic Cloud on Kubernetes extends Kubernetes orchestration capabilities to allow you to deploy and manage components of the Elastic Stack. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Enterprise is a self-managed orchestration platform for deploying and managing the Elastic Stack at scale. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Self-managed deployments are Elastic Stack deployments managed without the assistance of an orchestrator. Availability This functionality is generally available and ready for production usage. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
ECH
Generally available
ECK
Generally available
ECE
Generally available
Self-managed
Generally available
The number of shards on a node jumped sharply compared with the prior observation window, faster than normal growth for that cluster. Frequent index creation, high primary counts per index, daily rollovers, extra replicas, splits, or ingest spikes are common drivers.
For a complete list of insights, refer to AutoOps insights.
| Field | Value |
|---|---|
| Component | Elasticsearch |
| Severity | Medium |
| Scope | Index |
| Domains | index-management, shard-allocation |
You can customize these settings to adjust when AutoOps detects this event and presents the insight. Refer to AutoOps event settings for details.
The default customization settings are:
| Setting | Type | Default |
|---|---|---|
| Minimum shards count threshold | Integer | 500 |
| Minimum shards increase percentage | Percentage | 50 |
| Shard increase check period in hours | Integer | 12 |
| Minimum shards count increase | Integer | 50 |
The following is an example of what you might see when this insight is triggered. Real insights use live data and links from your deployment or cluster.
An unusual increase in the number of shards was identified on node es-data-01 and es-data-02. Over the past 48 hours, the number of shards has increased by 12%. However, in the previous period of 48 hours, the number of shards had increased only by 12%.
AutoOps shows different recommendations depending on how their conditions match your deployment or cluster.
Add data node
Condition: Shown when shard growth exceeds cluster capacity.
Add a data node to increase capacity and reduce pressure on the existing nodes.
Reduce shard count
Condition: Shown when index has high primary shard count.
Use the shrink API to reduce primary shards on logs-prod-000045 from 24 to 12 in logs-prod-000046. See the Shrink API for prerequisites.
POST logs-prod-000045/_shrink/logs-prod-000046?pretty{ "settings": { "index.number_of_shards": 12 } }
Requires the manage cluster privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
Reduce shard replicas
Condition: Shown when shard growth exceeds cluster capacity.
Reduce replica count on logs-prod-000045 from 2 to 1 to free disk space. Start with indices that have more than two shard copies.
PUT logs-prod-000045/_settings{ "index": { "number_of_replicas": 1 } }
Requires the manage index privilege. Requires Elasticsearch 8.0.0 or later. This action changes cluster or index configuration.
The sharp increase of shards can occur due to frequent index creation, high shard settings per index, daily time-based indices, rollover actions, excessive use of replicas, index splitting, increased data intake, or a lack of shard optimization or merging. A rapid, unsupervised rise in the number of shards can negatively impact performance.