Export failures when sending telemetry data from the Elastic Agent

During high traffic or load testing scenarios, the Elastic Agent might fail to export telemetry data (traces, metrics, or logs) to Elasticsearch. This typically happens when the internal queue for outgoing data fills up faster than it can be drained, resulting in timeouts and dropped data.

If you're experiencing network connectivity issues, refer to Connectivity issues. If no data appears in Kibana, refer to No data visible in Kibana.

Symptoms

You might see one or more of the following messages in the Elastic Agent logs:

bulk indexer flush error: failed to execute the request: context deadline exceeded
Exporting failed. Rejecting data. sending queue is full

These errors indicate the Collector is overwhelmed and unable to export data fast enough, leading to queue overflows and data loss.

Causes

This issue typically occurs when the sending_queue configuration or the Elasticsearch cluster scaling is misaligned with the incoming telemetry volume.

Important

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Deprecated since 9.3 This functionality is deprecated. You can still use it, but it'll be removed in a future Elastic Stack update. Generally available from 9.0 to 9.2 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. The sending queue is turned off by default. Verify that enabled: true is explicitly set — otherwise any queue configuration will be ignored.

Common contributing factors include:

Underscaled Elasticsearch cluster is the most frequent cause of persistent export failures. If Elasticsearch cannot index data fast enough, the Collector’s queue fills up.
The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Deprecated since 9.3 This functionality is deprecated. You can still use it, but it'll be removed in a future Elastic Stack update. Generally available from 9.0 to 9.2 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. sending_queue.block_on_overflow is turned off (defaults to false), which can lead to data drops.
Sending queue is enabled but num_consumers is too low to keep up with the incoming data volume.
Sending queue size (queue_size) is too small for the traffic load.
Both internal and sending queue batching are disabled, increasing processing overhead.
Elastic Agent resources (CPU, memory) are insufficient for the traffic volume.

Note

Increasing the timeout value (for example from 30s to 90s) doesn't help if the queue itself or Elasticsearch throughput is the bottleneck.

Resolution

The resolution approach depends on your Elastic Stack version and Collector configuration.

When the sending queue is not enabled by default

Enable the sending queue and block on overflow to prevent data drops:

sending_queue:
  enabled: true
  queue_size: 1000
  num_consumers: 10
  block_on_overflow: true

When the sending queue is enabled by default

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

The Elasticsearch exporter provides default sending_queue parameters (including block_on_overflow: true) but these can and often should be tuned for specific workloads.

The following steps can help identify and resolve export bottlenecks:

Check the Collector's internal metrics

If internal telemetry is enabled, review these metrics:
- otelcol.elasticsearch.bulk_requests.latency — high tail latency suggests Elasticsearch is the bottleneck. Check Elasticsearch cluster metrics and scale if necessary.
- otelcol.elasticsearch.bulk_requests.count and otelcol.elasticsearch.flushed.bytes — they help assess whether the Collector is sending too many or too large requests. Tune sending_queue.num_consumers or batching configuration to balance throughput.
- otelcol_exporter_queue_size and otelcol_exporter_queue_capacity — if the queue runs near capacity, but Elasticsearch is healthy, increase the queue size or number of consumers.
- otelcol_enqueue_failed_spans, otelcol_enqueue_failed_metric_points, otelcol_enqueue_failed_log_records — persistent enqueue failures indicate undersized queues or slow consumers.
For a complete list of available metrics, refer to the upstream OpenTelemetry metadata files for the Elasticsearch exporter and exporter helper.
Scale the Collector's resources
- Ensure sufficient CPU and memory for the Elastic Agent.
- Scale vertically (more resources) or horizontally (more replicas) as needed.
For Kubernetes deployments, refer to Insufficient resources in Kubernetes for detailed resource configuration guidance.
Optimize Elasticsearch performance

Address indexing delays, rejected bulk requests, or shard imbalances that limit ingestion throughput.

Tip

For more detailed diagnostics, refer to Enable debug logging to troubleshoot export failures.