Reindex a time series data stream

Turn on past index creation

Elasticsearch can create missing backing indices when you add data that precedes existing time ranges. To enable this feature, update the cluster settings:

PUT _cluster/settings
{
  "persistent": {
    "data_stream.past_tsdb_index_creation_enabled": true,
    "data_streams.past_tsdb_index_interval": "2d"
  }
}

1. These settings are persistent and apply to all data streams in the cluster.
2. By default, each past backing index covers one day of data. Refer to data_streams.past_tsdb_index_interval.

If you do not want to use this feature, you can use the steps provided for earlier versions, which include calculating time bounds and applying temporary index template settings.

Create the destination index template

Create an index template for the destination TSDS with the create an index template API. Use your preferred mappings and settings. For example:

PUT _index_template/my-new-tsds-template
{
  "index_patterns": ["my-new-tsds"],
  "priority": 100,
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series"
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "dimension_field": {
          "type": "keyword",
          "time_series_dimension": true
        },
        "metric_field": {
          "type": "double",
          "time_series_metric": "gauge"
        }
      }
    }
  }
}

Don't add data stream lifecycle or index lifecycle management details to the template yet. That keeps lifecycle actions such as downsampling from modifying the destination while you reindex.

Create the destination data stream

Create a data stream with the create a data stream API. The data stream name must match the index_patterns in your index template. For example:

PUT _data_stream/my-new-tsds

Run the reindex operation

Use the reindex API to copy documents from the source to the destination data stream. For example:

POST /_reindex
{
  "source": {
    "index": "my-old-tsds"
  },
  "dest": {
    "index": "my-new-tsds",
    "op_type": "create"
  }
}

Elasticsearch adds past backing indices to the destination data stream because you turned on past index creation.

Use the get data stream API to check that backing indices cover the timestamps you reindexed. For example:

GET _data_stream/my-new-tsds

The response lists each backing index and the time range it accepts.

Add data stream lifecycle

Optionally add a data stream lifecycle to manage retention, performance, and storage. For example, use the update data stream lifecycles API:

PUT _data_stream/my-new-tsds/_lifecycle
{
  "enabled": true,
  "data_retention": "365d"
}

1. The retention period of this data stream is set to 365 days. This means that Elasticsearch is allowed to delete data that is older than a year at its own discretion.

For more details, refer to Set a data stream's lifecycle.

Alternatively, you can use index lifecycle management. Refer to Create an index lifecycle management policy in Elasticsearch and Apply an index lifecycle policy to an existing Elasticsearch index.

Turn off past index creation

If you don't want the past index creation feature enabled for all of the data streams in your cluster, turn it off:

PUT _cluster/settings
{
  "persistent": {
    "data_stream.past_tsdb_index_creation_enabled": false
  }
}

Refer to data_streams.past_tsdb_index_interval.

Identify time bounds

Search for the lowest and highest @timestamp values in the old data stream. For example:

POST my-old-tsds/_search?size=0
{
  "aggs": {
    "min_ts": { "min": { "field": "@timestamp" } },
    "max_ts": { "max": { "field": "@timestamp" } }
  }
}

Add a buffer on both sides of these values. Subtract a small amount from the min date and add a small amount to the max date. For example, if your observed range is from 2026-10-13T17:50:23.747Z to 2026-10-13T18:50:21.747Z, use values like 2026-10-13T17:50:22.747Z and 2026-10-13T18:50:22.747Z when you create the destination index template.

Identify number of shards

Calculate the sum of all primary shards of all backing indices of the old data stream. For example, list the backing indices for the data stream with the get data streams API:

GET _data_stream/my-old-tsds

For each backing index, check its index.number_of_shards setting. For example:

GET .ds-my-old-tsds-*/_settings/index.number_of_shards

Sum theindex.number_of_shards values for all backing indices of the source data stream. This value is used when you create the destination index template.

Create the destination index template

Create an index template for the destination TSDS with the create an index template API. Use temporary settings so the first backing index can accept the source timestamps:

1. Set index.time_series.start_time and index.time_series.end_time to the time bounds you identified in the previous step.
2. Set index.number_of_shards to the number of shards you identified in the previous step.
3. Clear the index.lifecycle.name index setting (if any), to prevent index lifecycle management from modifying the destination data stream during reindexing.
4. (Optional) Set index.number_of_replicas to zero to speed up reindexing. Because the data is copied, you don't need replicas.

PUT _index_template/my-new-tsds-template
{
  "index_patterns": ["my-new-tsds"],
  "priority": 100,
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series",
      "index.time_series.start_time": "2023-01-01T00:00:00Z",
      "index.time_series.end_time": "2025-01-01T00:00:00Z",
      "index.number_of_shards": 6,
      "index.number_of_replicas": 0,
      "index.lifecycle.name": null
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "dimension_field": {
          "type": "keyword",
          "time_series_dimension": true
        },
        "metric_field": {
          "type": "double",
          "time_series_metric": "gauge"
        }
      }
    }
  }
}

1. Lowest timestamp value in the old data stream minus a small buffer
2. Highest timestamp value in the old data stream plus a small buffer
3. Sum of the primary shards from all source backing indices
4. Speed up reindexing
5. Pause index lifecycle management if it was configured

Create the destination data stream

Create a data stream with the create a data stream API. The data stream name must match the index_patterns in your index template. For example:

PUT _data_stream/my-new-tsds

Run the reindex operation

Use the reindex API to copy documents from the source to the destination data stream. For example:

POST /_reindex
{
  "source": {
    "index": "my-old-tsds"
  },
  "dest": {
    "index": "my-new-tsds",
    "op_type": "create"
  }
}

Use the get data stream API to check that backing indices cover the timestamps you reindexed. For example:

GET _data_stream/my-new-tsds

The response lists each backing index and the time range it accepts.

Restore the destination index template

After reindexing completes, update the index template to remove the temporary settings:

• Remove the overrides for index.time_series.start_time and index.time_series.end_time.
• Restore the values of index.number_of_shards, index.number_of_replicas, and index.lifecycle.name (as applicable).

PUT _index_template/my-new-tsds-template
{
  "index_patterns": ["my-new-tsds"],
  "priority": 100,
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series",
      "index.number_of_replicas": 1,
      "index.lifecycle.name": "my-ilm-policy"
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "dimension_field": {
          "type": "keyword",
          "time_series_dimension": true
        },
        "metric_field": {
          "type": "double",
          "time_series_metric": "gauge"
        }
      }
    }
  }
}

1. Restore replicas
2. Re-enable index lifecycle management if it was configured

Add data stream lifecycle

Optionally add a data stream lifecycle to manage retention, performance, and storage. For example, use the update data stream lifecycles API:

PUT _data_stream/my-new-tsds/_lifecycle
{
  "enabled": true,
  "data_retention": "365d"
}

1. The retention period of this data stream is set to 365 days. This means that Elasticsearch is allowed to delete data that is older than a year at its own discretion.

For more details, refer to Set a data stream's lifecycle.

Alternatively, you can use index lifecycle management. Refer to Create an index lifecycle management policy in Elasticsearch and Apply an index lifecycle policy to an existing Elasticsearch index.

Roll over for new data

The reindex operation copies documents into a single backing index of the destination data stream. Roll over the data stream so new documents go to a fresh backing index created with the intended lifecycle configuration. For example, use the roll over API:

POST my-new-tsds/_rollover/

When this step is complete, the destination data stream can accept new documents.

Identify time bounds

Search for the lowest and highest @timestamp values in the old data stream. For example:

POST my-old-tsds/_search?size=0
{
  "aggs": {
    "min_ts": { "min": { "field": "@timestamp" } },
    "max_ts": { "max": { "field": "@timestamp" } }
  }
}

Add a buffer on both sides of these values. Subtract a small amount from the min date and add a small amount to the max date. For example, if your observed range is from 2026-10-13T17:50:23.747Z to 2026-10-13T18:50:21.747Z, use values like 2026-10-13T17:50:22.747Z and 2026-10-13T18:50:22.747Z in the next step.

Create the destination index template

Create an index template for the destination TSDS with the create an index template API. Set index.time_series.start_time and index.time_series.end_time to the time bounds identified in the preceding step so the first backing index can accept the source timestamps. For example:

PUT _index_template/my-new-tsds-template
{
  "index_patterns": ["my-new-tsds"],
  "priority": 100,
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series",
      "index.time_series.start_time": "2023-01-01T00:00:00Z",
      "index.time_series.end_time": "2025-01-01T00:00:00Z",
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "dimension_field": {
          "type": "keyword",
          "time_series_dimension": true
        },
        "metric_field": {
          "type": "double",
          "time_series_metric": "gauge"
        }
      }
    }
  }
}

1. Lowest timestamp value in the old data stream
2. Highest timestamp value in the old data stream

Don't add optional data stream lifecycle details such as downsampling or data retention to the template yet. By using only the default configuration, you reduce the actions that can modify the destination while you reindex.

Create the destination data stream

Create a data stream with the create a data stream API. The data stream name must match the index_patterns in your index template. For example:

PUT _data_stream/my-new-tsds

Run the reindex operation

Use the reindex API to copy documents from the source to the destination data stream. For example:

POST /_reindex
{
  "source": {
    "index": "my-old-tsds"
  },
  "dest": {
    "index": "my-new-tsds",
    "op_type": "create"
  }
}

Documents land in that one backing index because its time range was set on the template.

Use the get data stream API to check that the backing index covers the timestamps you reindexed. For example:

GET _data_stream/my-new-tsds

The response lists each backing index and the time range it accepts.

Restore the destination index template

After reindexing completes, update the index template and remove index.time_series.start_time and index.time_series.end_time.

PUT _index_template/my-new-tsds-template
{
  "index_patterns": ["my-new-tsds"],
  "priority": 100,
  "data_stream": {},
  "template": {
    "settings": {
      "index.mode": "time_series"
    },
    "mappings": {
      "properties": {
        "@timestamp": {
          "type": "date"
        },
        "dimension_field": {
          "type": "keyword",
          "time_series_dimension": true
        },
        "metric_field": {
          "type": "double",
          "time_series_metric": "gauge"
        }
      }
    }
  }
}

Add data stream lifecycle

After the reindex completes, optionally customize the data stream lifecycle to manage retention, performance, and storage. For example, use the update data stream lifecycles API:

PUT _data_stream/my-new-tsds/_lifecycle
{
  "enabled": true,
  "data_retention": "365d"
}

1. The retention period of this data stream is set to 365 days. This means that Elasticsearch is allowed to delete data that is older than a year at its own discretion.

For more details, refer to Set a data stream's lifecycle.

Roll over for new data

The reindex operation copies documents into a single backing index of the destination data stream. Roll over the data stream so new documents go to a fresh backing index created with the intended lifecycle configuration. For example, use the roll over API:

POST my-new-tsds/_rollover/

When this step is complete, the destination data stream can accept new documents. Later rollovers happen automatically.