Type to search.

Quickstart: Ingest custom metrics with Elastic OpenTelemetry

Serverless Observability

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Elastic Agent

Elastic Agent can run in OpenTelemetry mode to retrieve traces, metrics, and logs from your infrastructure and applications, and forward them to Elastic Observability.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Use this quickstart to send custom metrics to Elastic using the Elastic Agent.

You’ll install a lightweight Elastic Agent, configure a minimal OpenTelemetry Protocol (OTLP) metrics pipeline, and verify the data in Elastic Observability.

  • An Elastic deployment (Serverless, Elastic Cloud Hosted, or self-managed)
  • An Observability project or Kibana instance
  • Permissions to create API keys
  • A system to run the Elastic Agent (Docker, host, or VM)
  • Optional: An application that emits OpenTelemetry metrics

  1. Create an Elastic API key

    In your Elastic Observability deployment:

    1. Go to Management → Stack Management → API keys.
    2. Create a new API key and copy the value.
    3. Note your deployment's OTLP ingest endpoint.

  2. Run the Elastic Agent with a minimal metrics pipeline

    Create a collector-config.yaml file with the following Collector configuration to receive OTLP metrics and export them to Elastic:

    YAML 
    receivers:
  otlp:
    protocols:
      http:
      grpc:

processors:
  batch: {}

exporters:
 otlphttp:
   endpoint: "<OTLP_ENDPOINT>"
   headers:
     Authorization: "ApiKey <YOUR_API_KEY>"

service:
 pipelines:
   metrics:
     receivers: [otlp]
     processors: [batch]
     exporters: [otlphttp]

    Run the configuration, for example with Docker:

    Shell 
    docker run --rm \
  -e ELASTIC_AGENT_OTEL=true \
  -v $(pwd)/collector-config.yaml:/etc/otel/config.yaml \
  -p 4317:4317 -p 4318:4318 \
  elastic/elastic-agent:9.5.6 \
  --config /etc/otel/config.yaml

  3. Optional: Port conflict handling

    If you encounter a port conflict error like:

    Shell 
    bind: address already in use

    Add the following to the service section:

    YAML 
    service:
 telemetry:
   metrics:
     address: localhost:8889
    1. Using different port if 8888 is already in use

    You can also verify if the Collector is listening on the correct ports:

    Shell 
    lsof -i :4318 -i :4317

  4. Send a custom metric

    In this Python example, you use an application that emits OTLP metrics. For other languages, refer to the contrib OpenTelemetry documentation.

    Python 
    from opentelemetry import metrics
from opentelemetry.exporter.otlp.proto.http.metric_exporter import OTLPMetricExporter
from opentelemetry.sdk.metrics import MeterProvider
from opentelemetry.sdk.metrics.export import PeriodicExportingMetricReader

exporter = OTLPMetricExporter(endpoint="http://localhost:4318/v1/metrics")
reader = PeriodicExportingMetricReader(exporter)
provider = MeterProvider(metric_readers=[reader])
metrics.set_meter_provider(provider)

meter = metrics.get_meter("custom-app")

temperature = meter.create_observable_gauge(
    "custom.temperature",
    callbacks=[lambda options: [metrics.Observation(26.7)]],
)

input("Sending metrics periodically... press Enter to stop")

  5. Verify metrics in Elastic Observability

    In Kibana:

    1. Find Discover in the navigation menu or use the global search field.

    2. Select ES|QL to switch to ES|QL mode, then run a TS query to select your metrics data:

      ES|QL 
      TS metrics-*

    3. Search the chart grid for custom.temperature, then break the metric down by dimension or add its chart to a dashboard.

    To learn how to explore metrics in Discover, refer to Explore metrics data with Discover in Kibana.

    1. Find Infrastructure in the navigation menu or use the global search field, then open Metrics Explorer.
    2. Search for custom.temperature.
    3. Visualize or aggregate the metric data.

    For the Metrics Explorer workflow, refer to Explore infrastructure metrics over time.

You've successfully set up a minimal OTLP metrics pipeline with the Elastic Agent. Your custom metrics are flowing into Elastic Observability and can be visualized in Kibana.

Now you can:

  • Build dashboards from your custom metrics
  • Set up alerts based on your custom metrics
  • Aggregate and analyze metric trends over time

You can expand your metrics collection setup in several ways:

  • Add more receivers to collect additional metrics
  • Configure the same Collector to send logs and traces alongside metrics

To learn more, refer to the Elastic Agent documentation.