Type to search.

Create a data stream with a lifecycle

Serverless

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Create a classic Elasticsearch data stream with a configured lifecycle. Learn how to set the retention period for your data and to retrieve the lifecycle configuration details.

For wired data streams, go to Get data into Streams and Configure data lifecycle with Streams.

To follow these steps, you need:

  • Cluster privilege manage_index_templates, to create the index template.

  • Index privileges:

    • create_index, create_doc, and view_index_metadata to create the stream, index documents, and read the lifecycle.
    • auto_configure if you use the bulk indexing API example, which adds mappings for fields that aren't defined in the template.
    • The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch projects. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. manage if you create the stream in Streams, which writes data stream settings.

    Refer to Granting privileges for data streams and aliases.

  • The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch projects. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. The All Kibana privilege for the Streams feature.

  • The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.5 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Elastic Cloud Serverless. A default snapshot repository and an appropriate license for searchable snapshots if you turn on frozen phase transitions. Refer to Manage snapshot repositories.

A data stream requires a matching index template. Set the lifecycle on that template, in the same place you set mappings and index settings, so every new stream that matches the pattern inherits the lifecycle details.

  1. Find Index Management in the navigation menu or use the global search field.

  2. On the Index Templates tab, click Create template.

  3. On the Logistics page, enter a name and an index pattern that matches the data stream. For example, use my-index-template for the name and my-data-stream-test* for the index pattern.

    The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch projects. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. To use the Streams app in subsequent steps, your index pattern must contain a wildcard (*).

  4. Turn on Create data stream.

  5. Select Delete phase (Data retention in earlier versions), and choose how long to retain your data, in days, hours, minutes, or seconds.

  6. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.5 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Elastic Cloud Serverless. Optional: Select the Frozen phase to choose when to move your data to searchable snapshots, in days, hours, minutes, or seconds. This period must be shorter than the delete phase. Refer to Searchable snapshots for data streams.

  7. In Priority, enter a value higher than 500 so it doesn't collide with built-in templates. Refer to Avoid index pattern collisions.

  8. Complete the rest of the wizard. On the Review page, click Create template. For the other wizard pages, refer to Index templates and Index fundamentals.

Use the create index template API. For example:

JSON
PUT _index_template/my-index-template
{
  "index_patterns": ["my-data-stream-test*"],
  "data_stream": { },
  "priority": 501,
  "template": { 
    "lifecycle": {
      "data_retention": "7d"
    }
  },
  "_meta": {
    "description": "Template with data stream lifecycle"
  }
}
  1. You can use a wildcard (*) in the index pattern to match every data stream whose name matches the pattern.
  2. Include the data_stream object to enable data streams.
  3. Set the priority higher than 500 so it doesn't collide with built-in templates. Refer to Avoid index pattern collisions.
  4. Define the lifecycle in the template section or in a component template.
  5. Choose how long to retain your data, in days, hours, minutes, or seconds.
Tip Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Elastic Cloud Serverless. The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.5 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

To move older backing indices to searchable snapshots, include frozen_after in the lifecycle on the template. For requirements and how conversion works, refer to Searchable snapshots for data streams. For example:

JSON
PUT _index_template/my-index-template
{
  "index_patterns": ["my-data-stream-test*"],
  "data_stream": { },
  "priority": 501,
  "template": {
    "lifecycle": {
      "data_retention": "365d",
      "frozen_after": "30d"
    }
  },
  "_meta": {
    "description": "Template with data stream lifecycle and frozen transitions"
  }
}
  1. This period must be shorter than the data_retention period if both are specified.

Create the data stream after the template exists. The stream name has to match the template's index pattern. Otherwise, the lifecycle on the template doesn't apply.

  • Create a data stream and load data by submitting API requests in Dev Tools. Refer to the API tab for examples.

  • The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch projects. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. Create a classic data stream in Streams:

    1. Find Streams in the navigation menu or use the global search field.
    2. Click Create classic stream.
    3. Select the index template, enter the stream name, and click Create.

Create the stream in either of these ways:

  • With the create data stream API. For example:

    JSON
    PUT _data_stream/my-data-stream-test

    The stream's name must match one of your template's index patterns.

  • With an indexing request. For example:

    JSON
    PUT my-data-stream-test/_bulk
    { "create":{ } }
    { "@timestamp": "2099-05-06T16:21:15.000Z", "message": "192.0.2.42 - - [06/May/2099:16:21:15 +0000] \"GET /images/bg.jpg HTTP/1.0\" 200 24736" }
    { "create":{ } }
    { "@timestamp": "2099-05-06T16:25:42.000Z", "message": "192.0.2.255 - - [06/May/2099:16:25:42 +0000] \"GET /favicon.ico HTTP/1.0\" 200 3638" }

Confirm that the data stream uses the lifecycle from the template.

  • View the data stream in Index Management:

    1. Find Index Management in the navigation menu or use the global search field.
    2. Select the Data Streams tab.
    3. Search for the data stream, and then select it.
    4. In the flyout, review the data phases (Data retention in earlier versions).
  • The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch projects. Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. View the data stream in Streams:

    1. Find Streams in the navigation menu or use the global search field.
    2. Select your stream from the list.
    3. Select the Data lifecycle tab (Retention in earlier versions).
    4. Review the lifecycle summary and data phases. For more information, refer to Configure data lifecycle with Streams.

Use the get data stream lifecycle API to view the lifecycle configuration details. For example:

JSON
GET _data_stream/my-data-stream-test*/_lifecycle

Result:

JSON
{
  "data_streams": [
    {
      "name": "my-data-stream-test",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d",
        "effective_retention": "7d",
        "retention_determined_by": "data_stream_configuration"
      }
    }
  ],
  "global_retention": {}
}
  1. The name of your data stream.
  2. Whether the data stream lifecycle is turned on for this data stream.
  3. The retention period configured for data in this data stream.
  4. The retention period that the data stream lifecycle uses. Elasticsearch keeps this data for at least 7 days and can delete it later.

Use the explain data stream lifecycle API to view lifecycle details for the backing indices. For example:

JSON
GET my-data-stream-test*/_lifecycle/explain

Result:

JSON
{
  "indices": {
    ".ds-my-data-stream-test-2026.10.23-000001": {
      "index": ".ds-my-data-stream-test-2026.10.23-000001",
      "managed_by_lifecycle": true,
      "index_creation_date_millis": 1792718438578,
      "time_since_index_creation": "22.79m",
      "lifecycle": {
        "enabled": true,
        "data_retention": "7d",
        "effective_retention": "7d",
        "retention_determined_by": "data_stream_configuration"
      }
    }
  }
}
  1. The name of the backing index.
  2. Whether the data stream lifecycle manages this index.
  3. Time since the index was created.
  4. The lifecycle configuration applied to this backing index.