Type to search.

Kafka partition metricset

Serverless

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

This is the partition metricset of the Kafka module.

Configuration

As the partition metricset fetches the data from the complete Kafka cluster, only one connection host has to be defined. Currently if multiple hosts are defined, the data is fetched multiple times. Support for multiple initial connections host is planned to be added in future releases.

Metricset

The current implementation of the partition metricset fetches the data for all leader partitions. Data for the replicas is not available yet.

This is a default metricset. If the host module is unconfigured, this metricset is enabled by default.

Fields

For a description of each field in the metricset, see the exported fields section.

Here is an example document generated by this metricset:

JSON 
{
    "@timestamp": "2017-10-12T08:05:34.853Z",
    "event": {
        "dataset": "kafka.partition",
        "duration": 115000,
        "module": "kafka"
    },
    "kafka": {
        "broker": {
            "address": "172.21.0.2:9092",
            "id": 0
        },
        "partition": {
            "id": 0,
            "offset": {
                "newest": 1,
                "oldest": 0
            },
            "partition": {
                "insync_replica": true,
                "is_leader": true,
                "leader": 0,
                "replica": 0
            },
            "topic_broker_id": "0-metricbeat-generate-data-0",
            "topic_id": "0-metricbeat-generate-data"
        },
        "topic": {
            "name": "metricbeat-generate-data"
        }
    },
    "metricset": {
        "name": "partition",
        "period": 10000
    },
    "service": {
        "address": "172.21.0.2:9092",
        "type": "kafka"
    }
}