Type to search.

Add an OTel Collector in Fleet

Serverless Preview

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Preview

    This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply.

Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

Stack Preview 9.4+

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Preview since 9.4

    This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Add an OpenTelemetry (OTel) Collector to Fleet for centralized monitoring using the Add collector flow. Fleet generates an OTel Collector configuration with the OpAMP extension wired to Fleet Server, an OTLP receiver, OTLP-to-Elasticsearch pipelines, and internal telemetry configuration. You then apply the configuration to your collector.

You'll need:

  • An Elastic Stack deployment version 9.4 or later or an Elastic Cloud Serverless Observability project

  • A Kibana user with the Admin role. For more information, refer to User roles and privileges.

  • An OTel Collector with OpAMP extension support. For supported distributions and versions, refer to Monitor OpenTelemetry Collectors in Fleet.

  • A running Fleet Server

    Note

    If you're using an Elastic Cloud Hosted deployment or a Serverless Observability project, Fleet Server is already available. For self-managed deployments, refer to Deploy on-premises and self-managed.

  1. Start adding a collector

    1. In Kibana, enter Fleet in the global search field, then select Fleet / Agents.
    2. Click Add, then select Collector (OpAMP) from the list.

    A flyout opens where you can enter metadata for your OTel Collector and preview the generated configuration.

  2. Enter collector metadata

    Provide values for the fields in the flyout. Fleet uses them to populate the OpAMP identity attributes and internal telemetry resource attributes in the generated configuration.

    Field Required Description
    Collector group display name Yes Human-readable label for this group of collectors (for example, Production West). Defaults to OTel Collector Group. Sets elastic.collector.group_name.
    Collector group Yes Identifier used for filtering collectors in the Fleet UI. Auto-derived from the group display name as a slug. If you override the value, use only lowercase letters, numbers, and hyphens. Sets elastic.collector.group and service.namespace.
    Service name Yes Identifier for the collector group in Elasticsearch. Auto-derived from the group display name as a slug. If you override the value, use only lowercase letters, numbers, and hyphens. Sets service.name.
    Collector display name Yes Per-instance identity that distinguishes this collector within the group. Defaults to ${env:HOSTNAME}. Sets elastic.display.name and service.instance.id.
    Config name No A short name for this collector configuration (for example, webserver-logs). Appears as the configuration label in the Fleet UI. Sets config.name.
    Config description No A human-readable summary of what the collector does. Appears as a comment header in the effective configuration view. Sets config.description.
    Tags No Comma-separated labels (for example, prod,west-region,k8s). Tags appear in the Fleet UI tag filter and as resource attributes on self-emitted metrics and logs.
    Environment No Label for the deployment environment (for example, production or staging). Sets deployment.environment.name.

    A generated configuration preview appears when all required fields contain valid values.

    Note

    If you override the Collector group or Service name fields, they stop auto-updating from the group display name.

  3. Supply an Elasticsearch API key

    Note

    This applies to the elasticsearch/otel exporter included in the generated configuration. If your collector already exports to Elasticsearch with a valid API key, skip this step.

    Choose how to provide an Elasticsearch API key for the elasticsearch/otel exporter:

    • Create one in the flyout: Click Create API key. Fleet creates an API key with default privileges and substitutes it for ${API_KEY} in the generated configuration.
    • Use an existing key: Skip this action in the flyout, and replace ${API_KEY} manually after copying the configuration in the next step. The key must have create_index, write, and auto_configure index privileges on metrics-*, logs-*, and traces-* data streams.

    Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Preview This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Serverless interfaces and procedures might differ from classic Elastic Stack deployments. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Preview since 9.5 This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. If your deployment sends data through the Elastic Cloud Managed OTLP Endpoint, this step asks for a managed OTLP API key instead. Replace ${API_KEY} in the Authorization header rather than the api_key field, and note that the generated configuration exports through an otlp/managed exporter instead of elasticsearch/otel.

  4. Apply the configuration

    1. Copy the YAML configuration displayed in the flyout, or click Download config to save it as a YAML file. The snippet wires the OpAMP extension to Fleet Server, sets up an OTLP receiver and OTLP-to-Elasticsearch pipelines, and adds internal telemetry configuration.

      Note

      If you already have a working OTel Collector with an Elasticsearch exporter, merge the generated configuration into your existing setup instead of replacing the whole file. Remove the elasticsearch/otel exporter block and adjust the pipelines to use your existing exporter.

      Generated OTel Collector configuration
      YAML
      extensions:
        opamp:
          server:
            http:
              endpoint: "https://<fleet-server-host-url>/v1/opamp"
              headers:
                Authorization: "ApiKey <fleet-enrollment-api-key>"
              tls:
                insecure_skip_verify: true
          instance_uid: "<instance-uid>"
          agent_description:
            non_identifying_attributes:
              elastic.collector.group_name: "OTel Collector Group"
              elastic.collector.group: "otel-collector-group"
              elastic.display.name: "${env:HOSTNAME}"
      
      receivers:
        otlp:
          protocols:
            grpc:
              endpoint: "0.0.0.0:4317"
      
      exporters:
        elasticsearch/otel:
          endpoints:
            - "https://<elasticsearch-host-url>"
          api_key: "${API_KEY}"
          mapping:
            mode: otel
        otlp:
          endpoint: "http://localhost:4317"
          tls:
            insecure: true
      
      service:
        extensions: [opamp]
        pipelines:
          logs:
            receivers: [otlp]
            exporters: [elasticsearch/otel]
          metrics:
            receivers: [otlp]
            exporters: [elasticsearch/otel]
          traces:
            receivers: [otlp]
            exporters: [elasticsearch/otel]
        telemetry:
          resource:
            elastic.collector.group_name: "OTel Collector Group"
            elastic.collector.group: "otel-collector-group"
            service.namespace: "otel-collector-group"
            service.name: "otel-collector-group"
            service.instance.id: "${env:HOSTNAME}"
          metrics:
            readers:
              - periodic:
                  exporter:
                    otlp:
                      protocol: grpc
                      endpoint: "http://localhost:4317"
          logs:
            processors:
              - batch:
                  exporter:
                    otlp:
                      protocol: grpc
                      endpoint: "http://localhost:4317"
          traces:
            processors:
              - batch:
                  exporter:
                    otlp:
                      protocol: grpc
                      endpoint: "http://localhost:4317"
      
      1. The Fleet Server host URL with the OpAMP endpoint, automatically populated by Fleet.
      2. An enrollment API key, automatically populated by Fleet.
      3. Skips verification of the Fleet Server TLS certificate. Before deploying to production, adjust the configuration to use a CA file as described in Configure TLS for Fleet Server connection.
      4. A UUID v7 instance identifier, automatically generated by Fleet.
      5. Identity attributes populated from the form fields you provided in the previous step.
      6. The Elasticsearch endpoint, automatically populated from your default Fleet output.
      7. The Elasticsearch API key. If you used Create API key, Fleet replaces ${API_KEY} with the generated key. Otherwise, replace ${API_KEY} manually with your existing encoded key.
    2. Paste or merge the configuration into your OTel Collector configuration file (for example, otel.yaml).

    3. Make sure every environment-variable placeholder in the configuration resolves at runtime:

      • ${API_KEY}: replace it with your encoded Elasticsearch API key value, or set the API_KEY environment variable before starting or restarting the collector.
      • ${env:HOSTNAME}: make sure HOSTNAME is set in the collector's runtime environment, or replace the placeholder with a static identifier.
    4. Save your configuration.

  5. Start the collector

    Run the appropriate command for your collector type, pointing to the configuration file you saved in the previous step:

    Shell
    ./otelcol --config ./otel-opamp.yaml
    
    Shell
    ./otelcol-contrib --config ./otel-opamp.yaml
    

    Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Preview This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Serverless interfaces and procedures might differ from classic Elastic Stack deployments. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Preview since 9.5 This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. The flyout shows the same commands. Select Elastic Agent or OTel Contrib Collector and the command updates to match your choice.

  6. Verify the collector connection

    1. Confirm the collector started without errors in its output.
    2. Return to the Fleet UI. The flyout displays a confirmation message when your collector successfully connects.
    3. Your OTel Collector now appears in the Agents list. To go straight there, click View connected collectors.
Important

The generated configuration uses tls.insecure_skip_verify: true for the Fleet Server connection. Before deploying to production, adjust the configuration to use a CA file as described in Configure TLS for Fleet Server connection.

Note

On Serverless Observability projects and Elastic Cloud Hosted deployments, you can replace the elasticsearch/otel exporter with the Elastic Cloud Managed OTLP Endpoint, which accepts OTLP data directly and uses an APM-scoped API key. For more details, refer to Elastic Cloud Managed OTLP Endpoint (mOTLP).

If your Fleet Server uses a self-signed certificate or a certificate from a non-public Certificate Authority (CA), you need to configure the OpAMP extension to trust it.

When Fleet Server uses a certificate signed by a private CA, provide the CA certificate to your OTel Collector:

YAML
extensions:
  opamp:
    server:
      http:
        endpoint: https://fleet-server:8220/v1/opamp
        tls:
          ca_file: /path/to/ca.crt
        headers:
          Authorization: ApiKey <fleet-enrollment-api-key>
    instance_uid: <instance-uid>

service:
  extensions: [opamp]
  1. Replace /path/to/ca.crt with the path to your CA certificate file.

For testing purposes only, you can skip TLS certificate verification:

YAML
extensions:
  opamp:
    server:
      http:
        endpoint: https://fleet-server:8220/v1/opamp
        tls:
          insecure_skip_verify: true
        headers:
          Authorization: ApiKey <fleet-enrollment-api-key>
    instance_uid: <instance-uid>

service:
  extensions: [opamp]
  1. Set to true to skip TLS certificate verification.
Warning

Using insecure_skip_verify: true skips TLS certificate verification and makes your connection vulnerable to man-in-the-middle attacks. Only use this for testing in isolated environments, never in production.