Elasticsearch module

Prefer to use Elastic Agent for this use case?

Refer to the Elastic Integrations documentation.

Learn more

Elastic Agent is a single, unified way to add monitoring for logs, metrics, and other types of data to a host. It can also protect hosts from security threats, query data from operating systems, forward data from remote services or hardware, and more. Refer to the documentation for a detailed comparison of Beats and Elastic Agent.

The elasticsearch module collects metrics about Elasticsearch.

Compatibility

The elasticsearch module works with Elasticsearch 6.7.0 and later.

Usage for Elastic Stack Monitoring

The elasticsearch module can be used to collect metrics shown in our Stack Monitoring UI in Kibana. To enable this usage, set xpack.enabled: true and remove any metricsets from the module’s configuration. Alternatively, run metricbeat modules disable elasticsearch and metricbeat modules enable elasticsearch-xpack.

When xpack mode is enabled, all the legacy metricsets are automatically enabled by default. This means that you do not need to manually enable them, and there will be no conflicts or issues because Metricbeat merges the user-defined metricsets with the ones that xpack mode forces to enable. As a result, you can seamlessly collect comprehensive metrics without worrying about dataset overlap or duplication.

metricbeat.modules:
- module: elasticsearch
  xpack.enabled: true
  metricsets:
    - ingest_pipeline
  period: 10s

Note

When this module is used for Elastic Stack Monitoring, it sends metrics to the monitoring index instead of the default index typically used by Metricbeat. For more details about the monitoring index, see Configuring indices for monitoring.

Module-specific configuration notes

Like other Metricbeat modules, the elasticsearch module accepts a hosts configuration setting. This setting can contain a list of entries. The related scope setting determines how each entry in the hosts list is interpreted by the module.

If scope is set to node (default), each entry in the hosts list indicates a distinct node in an Elasticsearch cluster.
If scope is set to cluster, each entry in the hosts list indicates a single endpoint for a distinct Elasticsearch cluster (for example, a load-balancing proxy fronting the cluster).

Also like some other modules, the elasticsearch module accepts either a username/password pair of settings or a single api_key setting. You cannot specify the username/password settings and api_key at the same time.

When used, the api_key configuration can be specified as:

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.1 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. The unencoded id:api_key format (api_key: "foo:bar") or the Base64-encoded id:api_key format (api_key: "Zm9vOmJhcgo=").
All earlier releases can only use the unencoded id:api_key format (api_key: "foo:bar").

Example configuration

The Elasticsearch module supports the standard configuration options that are described in Modules. Here is an example configuration:

metricbeat.modules:
- module: elasticsearch
  metricsets:
    - node
    - node_stats
    #- index
    #- index_recovery
    #- index_summary
    #- ingest_pipeline
    #- shard
    #- ml_job
  period: 10s
  hosts: ["http://localhost:9200"]
  #username: "elastic"
  #password: "changeme"
  #api_key: "foo:bar"
  #ssl.certificate_authorities: ["/etc/pki/root/ca.pem"]

  #index_recovery.active_only: true
  #ingest_pipeline.processor_sample_rate: 0.25
  #xpack.enabled: false
  #scope: node

This module supports TLS connections when using ssl config field, as described in SSL. It also supports the options described in Standard HTTP config options.

Metricsets

The following metricsets are available:

ccr
cluster_stats
enrich
index
index_recovery
index_summary
ingest_pipeline The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Beta This functionality is in beta and is not ready for production usage. For beta features, the design and code is less mature than official GA features and is being provided as-is with no warranties. Beta features are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
ml_job
node
node_stats
pending_tasks
security_stats The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.2 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
shard