Type to search.

Troubleshoot incomplete migration to data tiers

Stack

The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.

Availability

  • Generally available

    This functionality is generally available and ready for production usage.

Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.

This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

Elasticsearch standardized the implementation of hot-warm-cold architectures to data tiers in version 7.10. Some indices and deployments might have not fully transitioned to data tiers and mix the new way of implementing the hot-warm-cold architecture with legacy based node attributes.

This could lead to unassigned shards or shards not transitioning to the desired tier.

To fix this issue, use the following steps.

You can run the following steps using either API console or direct Elasticsearch API calls.

Tip

To learn how to assign tiers to your data nodes, refer to Migrate index allocation filters to ILM node roles.

To get the shards assigned, call the migrate to data tiers routing API, which resolves the conflicting routing configurations to using the standardized data tiers. This also future-proofs the system by migrating the index templates and ILM policies if needed.

  1. Stop index lifecycle management

    JSON
    POST /_ilm/stop

    The response will look like this:

    JSON
    {
      "acknowledged": true
    }
    
  2. Wait for index lifecycle management to stop. Check the status until it returns STOPPED as follows:

    JSON
    GET /_ilm/status

    When index lifecycle management has successfully stopped the response will look like this:

    JSON
    {
      "operation_mode": "STOPPED"
    }
    
  3. Migrate to data tiers

    JSON
    POST /_ilm/migrate_to_data_tiers

    The response will look like this:

    JSON
    {
      "dry_run": false,
      "migrated_ilm_policies":["policy_with_allocate_action"],
      "migrated_indices":["warm-index-to-migrate-000001"],
      "migrated_legacy_templates":["a-legacy-template"],
      "migrated_composable_templates":["a-composable-template"],
      "migrated_component_templates":["a-component-template"]
    }
    
    1. The ILM policies that were updated.
    2. The indices that were migrated to tier preference routing.
    3. The legacy index templates that were updated to not contain custom routing settings for the provided data attribute.
    4. The composable index templates that were updated to not contain custom routing settings for the provided data attribute.
    5. The component templates that were updated to not contain custom routing settings for the provided data attribute.
  4. Restart index lifecycle management:

    JSON
    POST /_ilm/start

    The response will look like this:

    JSON
    {
      "acknowledged": true
    }