AuthThe Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.AvailabilityGenerally available since 8.12This functionality is generally available and ready for production usage.Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Create an inference endpoint to perform an inference task with the openai service or openai compatible APIs.
The type of the inference task that the model will perform.
NOTE: The chat_completion task type only supports streaming and only through the _stream API.
The maximum size of a chunk in words.
This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
overlapnumber · default: 100
The number of overlapping words for chunks.
It is applicable only to a word chunking strategy.
This value cannot be higher than half the max_chunk_size value.
sentence_overlapnumber · default: 1
The number of overlapping sentences for chunks.
It is applicable only for a sentence chunking strategy.
It can be either 1 or 0.
strategystring · default: sentence
The chunking strategy: sentence or word.
serviceenumrequired
The type of service supported for the specified task type. In this case, openai.
Values:openai
service_settingsobjectrequired
Settings used to install the inference model. These settings are specific to the openai service.
api_keystringrequired
A valid API key of your OpenAI account.
You can find your OpenAI API keys in your OpenAI account under the API keys section.
Important
You need to provide the API key only once, during the inference model creation.
The get inference endpoint API does not retrieve your API key.
The number of dimensions the resulting output embeddings should have.
It is supported only in text-embedding-3 and later models.
If it is not set, the OpenAI defined default for the model is used.
model_idstringrequired
The name of the model to use for the inference task.
Refer to the OpenAI documentation for the list of available text embedding models.
The unique identifier for your organization.
You can find the Organization ID in your OpenAI account under Settings > Organizations.
rate_limitobject
This setting helps to minimize the number of rate limit errors returned from OpenAI.
The openai service sets a default number of requests allowed per minute depending on the task type.
For text_embedding, it is set to 3000.
For completion, it is set to 500.
requests_per_minutenumber
The number of requests allowed per minute.
similarityenum
For a text_embedding task, the similarity measure. One of cosine, dot_product, l2_norm. Defaults to dot_product.
The maximum size of a chunk in words.
This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
overlapnumber · default: 100
The number of overlapping words for chunks.
It is applicable only to a word chunking strategy.
This value cannot be higher than half the max_chunk_size value.
sentence_overlapnumber · default: 1
The number of overlapping sentences for chunks.
It is applicable only for a sentence chunking strategy.
It can be either 1 or 0.
strategystring · default: sentence
The chunking strategy: sentence or word.
servicestringrequired
The service type
service_settingsobjectrequired
Settings specific to the service
task_settingsobject
Task settings specific to the service and task type