Create an Elasticsearch inference endpoint
- PUT /_inference/{task_type}/{elasticsearch_inference_id}
Create an inference endpoint to perform an inference task with the elasticsearch service.
Your Elasticsearch deployment contains preconfigured ELSER and E5 inference endpoints, you only need to create the enpoints using the API if you want to customize the settings.
If you use the ELSER or the E5 model through the elasticsearch service, the API request will automatically download and deploy the model if it isn't downloaded yet.
You might see a 502 bad gateway error in the response when using the Kibana Console. This error usually just reflects a timeout, while the model downloads in the background. You can check the download progress in the Machine Learning UI. If using the Python client, you can set the timeout parameter to a higher value.
After creating the endpoint, wait for the model deployment to complete before using it.
To verify the deployment status, use the get trained model statistics API.
Look for "state": "fully_allocated" in the response and ensure that the "allocation_count" matches the "target_allocation_count".
Avoid creating multiple endpoints for the same model unless required, as each endpoint consumes significant resources.
Required privileges
- Cluster privileges manage_inference
Request body
-
chunking_settingsobject -
The chunking configuration object. Applies only to the
sparse_embeddingandtext_embeddingtask types. Not applicable to thereranktask type. - Reference
-
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be lower than
20(forsentencestrategy) or10(forwordstrategy). This value should not exceed the window size for the associated model. -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
separator_groupstring -
Only applicable to the
recursivestrategy and required when using it.Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be
markdownorplaintext.Using this parameter is an alternative to manually specifying a custom
separatorslist. -
separatorsstring[] -
Only applicable to the
recursivestrategy and required when using it.A list of strings used as possible split points when chunking text.
Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the
max_chunk_sizelimit, to reduce the total number of chunks generated. -
strategystring · default: sentence -
The chunking strategy:
sentence,word,noneorrecursive.- If
strategyis set torecursive, you must also specify:
max_chunk_size- either
separatorsorseparator_group
Learn more about different chunking strategies in the linked documentation.
- If
- Reference
-
serviceenumrequired -
The type of service supported for the specified task type. In this case,
elasticsearch. -
Values:
elasticsearch -
service_settingsobjectrequired -
Settings used to install the inference model. These settings are specific to the
elasticsearchservice. -
adaptive_allocationsobject -
Adaptive allocations configuration details. If
enabledis true, the number of allocations of the model is set based on the current load the process gets. When the load is high, a new model allocation is automatically created, respecting the value ofmax_number_of_allocationsif it's set. When the load is low, a model allocation is automatically removed, respecting the value ofmin_number_of_allocationsif it's set. Ifenabledis true, do not set the number of allocations manually. -
deployment_idstring -
The deployment identifier for a trained model deployment. When
deployment_idis used themodel_idis optional. -
model_idstringrequired -
The name of the model to use for the inference task. It can be the ID of a built-in model (for example,
.multilingual-e5-smallfor E5) or a text embedding model that was uploaded by using the Eland client. - Reference
-
num_allocationsnumber -
The total number of allocations that are assigned to the model across machine learning nodes. Increasing this value generally increases the throughput. If adaptive allocations are enabled, do not set this value because it's automatically set.
-
num_threadsnumberrequired -
The number of threads used by each model allocation during inference. This setting generally increases the speed per inference request. The inference process is a compute-bound process;
threads_per_allocationsmust not exceed the number of available allocated processors per node. The value must be a power of 2. The maximum value is 32. -
long_document_strategystringThe Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Experimental This functionality is experimental and is not ready for production usage. Experimental features may change or be removed at any time. Elastic will work to fix any issues, but experimental features are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. -
Available only for the
reranktask type using the Elastic reranker model. Controls the strategy used for processing long documents during inference.Possible values:
truncate(default): Processes only the beginning of each document.chunk: Splits long documents into smaller parts (chunks) before inference.
When
long_document_strategyis set tochunk, Elasticsearch splits each document into smaller parts but still returns a single score per document. That score reflects the highest relevance score among all chunks. -
max_chunks_per_docnumberThe Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Experimental This functionality is experimental and is not ready for production usage. Experimental features may change or be removed at any time. Elastic will work to fix any issues, but experimental features are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. -
Only for the
reranktask type. Limits the number of chunks per document that are sent for inference when chunking is enabled. If not set, all chunks generated for the document are processed.
Response
-
chunking_settingsobject -
The chunking configuration object. Applies only to the
embedding,sparse_embeddingandtext_embeddingtask types. Not applicable to thererank,completion, orchat_completiontask types. -
max_chunk_sizenumber · default: 250 -
The maximum size of a chunk in words. This value cannot be lower than
20(forsentencestrategy) or10(forwordstrategy). This value should not exceed the window size for the associated model. -
overlapnumber · default: 100 -
The number of overlapping words for chunks. It is applicable only to a
wordchunking strategy. This value cannot be higher than half themax_chunk_sizevalue. -
sentence_overlapnumber · default: 1 -
The number of overlapping sentences for chunks. It is applicable only for a
sentencechunking strategy. It can be either1or0. -
separator_groupstring -
Only applicable to the
recursivestrategy and required when using it.Sets a predefined list of separators in the saved chunking settings based on the selected text type. Values can be
markdownorplaintext.Using this parameter is an alternative to manually specifying a custom
separatorslist. -
separatorsstring[] -
Only applicable to the
recursivestrategy and required when using it.A list of strings used as possible split points when chunking text.
Each string can be a plain string or a regular expression (regex) pattern. The system tries each separator in order to split the text, starting from the first item in the list.
After splitting, it attempts to recombine smaller pieces into larger chunks that stay within the
max_chunk_sizelimit, to reduce the total number of chunks generated. -
strategystring · default: sentence -
The chunking strategy:
sentence,word,noneorrecursive.- If
strategyis set torecursive, you must also specify:
max_chunk_size- either
separatorsorseparator_group
Learn more about different chunking strategies in the linked documentation.
- If
- Reference