Logs data streams

Logs data streams store log data more efficiently. In benchmarks, logsdb index mode reduced the storage footprint of log data by up to 60%, with a small impact (10-20%) to indexing performance. Results vary depending on your data set and Elasticsearch version.

Logs data streams are created when the index.mode in the relevant template is set to logsdb, either automatically or manually.

Logsdb index mode is enabled by default for logs in Elasticsearch, Observability, and Security Serverless projects, and for new logs data streams in Elastic Stack 9.0 and later.

Note

Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Unavailable This functionality is not available in Serverless Elasticsearch Vector Database projects. For a fully columnar logs storage profile, you can set index.mode to logsdb_columnar instead. To learn when to choose that mode, refer to Columnar index mode.

Availability of logsdb index mode

Logsdb index mode is automatically enabled for the following data streams:

Elasticsearch, Observability, and Security Serverless projects: Logsdb mode is automatically set on new and existing data streams with names matching the logs-*-* pattern.
Elastic Stack: Automatic logsdb mode depends on your version and configuration:
- As of Elasticsearch version 9.0, logsdb mode is automatically set on new data streams with names matching the logs-*-* pattern.
- In clusters that were upgraded from 8.x to 9.x:
  - If the instance had no existing logs-*-* data streams when you upgraded, new logs-*-* data streams are set to logsdb mode.
  - Existing data streams, including those used for integrations or APM in 8.x instances, are not automatically set to logsdb mode.

You can enable logsdb on existing data streams as needed, by editing the relevant index templates. For integrations, use @custom component templates.

Enable logsdb index mode

In most cases, you won't need to enable logsdb mode manually. If you do need to enable it, you can either create a new template or update an existing one.

Set index.mode to logsdb in the relevant index template or @custom component template:

New data streams: Create a new template, specifying logsdb for the index mode. New data streams matching the template's index pattern use logsdb mode automatically.
Existing data streams: Update the templates the data stream references. Logsdb mode will take effect on the next rollover. For integrations, refer to Enable logsdb for integrations.

Set logsdb mode in an index template

Kibana

To create or edit an index template in Kibana:

Go to Index Management using the navigation menu or the global search field.
On the Index Templates tab, click Create template to create a new template, or click the name of an existing template to edit it.

Tip

If you have existing templates labeled Managed, refer to Enable logsdb for integrations or Default logs index template.
Complete the steps in the wizard. In the Logistics step:
- Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Generally available This functionality is generally available and ready for production usage. Serverless interfaces and procedures might differ from classic Elastic Stack deployments. Click the Set index mode toggle to show the Index mode field.
- Select LogsDB as the Index mode.

API

To create or update an index template:

In an Elastic Stack deployment, use the create index template API.
In Elastic Cloud Serverless, use the create index template API.

First retrieve the current configuration, if any, so you can preserve existing settings. The PUT request overwrites any existing template.

GET _index_template/my-index-template

Set index.mode to logsdb in the template request:

PUT _index_template/my-index-template
{
  "index_patterns": ["my-data-*"],
  "data_stream": { },
  "template": {
     "settings": {
        "index.mode": "logsdb"
     }
  },
  "priority": 200
}

Set logsdb mode for integrations

To enable logsdb mode for integration data streams, create or update a @custom component template for each logs dataset. For details, refer to Enable logsdb for integrations.