AuthThe Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.AvailabilityGenerally available since 8.14This functionality is generally available and ready for production usage.Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Create an inference endpoint to perform an inference task with the azureopenai service.
The list of chat completion models that you can choose from in your Azure OpenAI deployment include:
The type of the inference task that the model will perform.
NOTE: The chat_completion task type only supports streaming and only through the _stream API.
The maximum size of a chunk in words.
This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
overlapnumber · default: 100
The number of overlapping words for chunks.
It is applicable only to a word chunking strategy.
This value cannot be higher than half the max_chunk_size value.
sentence_overlapnumber · default: 1
The number of overlapping sentences for chunks.
It is applicable only for a sentence chunking strategy.
It can be either 1 or 0.
strategystring · default: sentence
The chunking strategy: sentence or word.
serviceenumrequired
The type of service supported for the specified task type. In this case, azureopenai.
Values:azureopenai
service_settingsobjectrequired
Settings used to install the inference model. These settings are specific to the azureopenai service.
api_keystring
A valid API key for your Azure OpenAI account.
You must specify either api_key or entra_id.
If you do not provide either or you provide both, you will receive an error when you try to create your model.
Important
You need to provide the API key only once, during the inference model creation.
The get inference endpoint API does not retrieve your API key.
The Azure API version ID to use.
It is recommended to use the latest supported non-preview version.
deployment_idstringrequired
The deployment name of your deployed models.
Your Azure OpenAI deployments can be found though the Azure OpenAI Studio portal that is linked to your subscription.
A valid Microsoft Entra token.
You must specify either api_key or entra_id.
If you do not provide either or you provide both, you will receive an error when you try to create your model.
This setting helps to minimize the number of rate limit errors returned from Azure.
The azureopenai service sets a default number of requests allowed per minute depending on the task type.
For text_embedding, it is set to 1440.
For completion, it is set to 120.
The maximum size of a chunk in words.
This value cannot be higher than 300 or lower than 20 (for sentence strategy) or 10 (for word strategy).
overlapnumber · default: 100
The number of overlapping words for chunks.
It is applicable only to a word chunking strategy.
This value cannot be higher than half the max_chunk_size value.
sentence_overlapnumber · default: 1
The number of overlapping sentences for chunks.
It is applicable only for a sentence chunking strategy.
It can be either 1 or 0.
strategystring · default: sentence
The chunking strategy: sentence or word.
servicestringrequired
The service type
service_settingsobjectrequired
Settings specific to the service
task_settingsobject
Task settings specific to the service and task type