Inference
Inference APIs enable you to use certain services, such as built-in machine learning models (ELSER, E5), models uploaded through Eland, Cohere, OpenAI, Azure, Google AI Studio or Hugging Face. For built-in models and models uploaded through Eland, the inference APIs offer an alternative way to use and manage trained models. However, if you do not plan to use the inference APIs to use these models or if you want to use non-NLP models, use the machine learning trained model APIs.
-
Get the inference region policy
Stack
9.5+
- GET /_inference/_region_policy
-
Create or update the inference region policy
Stack
9.5+
- PUT /_inference/_region_policy
-
Delete the inference region policy
Stack
9.5+
- DELETE /_inference/_region_policy
-
Perform non-streaming chat completion inference on the service
Stack
Planned
- POST /_inference/chat_completion/{inference_id}
-
Perform streaming chat completion inference on the service
Stack
8.18+
- POST /_inference/chat_completion/{inference_id}/_stream
-
Perform completion inference on the service
Stack
8.11+
- POST /_inference/completion/{inference_id}
-
Perform streaming completion inference on the service
Stack
8.16+
- POST /_inference/completion/{inference_id}/_stream
-
Perform decision inference on the service
Stack
Planned
- POST /_inference/decision/{inference_id}
-
Perform document extraction inference on the service
Stack
Planned
- POST /_inference/document_extraction/{inference_id}
-
Perform dense embedding inference on the service
Stack
Experimental
9.4+
- POST /_inference/embedding/{inference_id}
-
Perform reranking inference on the service
Stack
8.11+
- POST /_inference/rerank/{inference_id}
-
Perform sparse embedding inference on the service
Stack
8.11+
- POST /_inference/sparse_embedding/{inference_id}
-
Perform text embedding inference on the service
Stack
8.11+
- POST /_inference/text_embedding/{inference_id}
-
Create a AI21 inference endpoint
Stack
9.2+
- PUT /_inference/{task_type}/{ai21_inference_id}
-
Create an AlibabaCloud AI Search inference endpoint
Stack
8.16+
- PUT /_inference/{task_type}/{alibabacloud_inference_id}
-
Create an Amazon Bedrock inference endpoint
Stack
8.12+
- PUT /_inference/{task_type}/{amazonbedrock_inference_id}
-
Create an Amazon SageMaker inference endpoint
Stack
9.1+
- PUT /_inference/{task_type}/{amazonsagemaker_inference_id}
-
Create an Anthropic inference endpoint
Stack
8.16+
- PUT /_inference/{task_type}/{anthropic_inference_id}
-
Create an Azure AI studio inference endpoint
Stack
8.14+
- PUT /_inference/{task_type}/{azureaistudio_inference_id}
-
Create an Azure OpenAI inference endpoint
Stack
8.14+
- PUT /_inference/{task_type}/{azureopenai_inference_id}
-
Create a Cohere inference endpoint
Stack
8.13+
- PUT /_inference/{task_type}/{cohere_inference_id}
-
Create an Contextual AI inference endpoint
Stack
9.2+
- PUT /_inference/{task_type}/{contextualai_inference_id}
-
Create a custom inference endpoint
Stack
8.19+
- PUT /_inference/{task_type}/{custom_inference_id}
-
Create a DeepSeek inference endpoint
Stack
9.1+
- PUT /_inference/{task_type}/{deepseek_inference_id}
-
Create an Elasticsearch inference endpoint
Stack
8.13+
- PUT /_inference/{task_type}/{elasticsearch_inference_id}
-
Create an ELSER inference endpoint
Stack
8.11+
Deprecated
- PUT /_inference/{task_type}/{elser_inference_id}
-
Create a Fireworks AI inference endpoint
Stack
9.4+
- PUT /_inference/{task_type}/{fireworksai_inference_id}
-
Create an Google AI Studio inference endpoint
Stack
8.15+
- PUT /_inference/{task_type}/{googleaistudio_inference_id}
-
Create a Google Vertex AI inference endpoint
Stack
8.15+
- PUT /_inference/{task_type}/{googlevertexai_inference_id}
-
Create a Groq inference endpoint
Stack
9.3+
- PUT /_inference/{task_type}/{groq_inference_id}
-
Create a Hugging Face inference endpoint
Stack
8.12+
- PUT /_inference/{task_type}/{huggingface_inference_id}
-
Get an inference endpoint
Stack
8.11+
- GET /_inference/{task_type}/{inference_id}
-
Perform inference on the service
Stack
8.11+
- POST /_inference/{task_type}/{inference_id}
-
Create an inference endpoint
Stack
8.11+
- PUT /_inference/{task_type}/{inference_id}
-
Delete an inference endpoint
Stack
8.11+
- DELETE /_inference/{task_type}/{inference_id}
-
Update an inference endpoint
Stack
8.17+
- PUT /_inference/{task_type}/{inference_id}/_update
-
Create an JinaAI inference endpoint
Stack
8.18+
- PUT /_inference/{task_type}/{jinaai_inference_id}
-
Create a Llama inference endpoint
Stack
9.2+
- PUT /_inference/{task_type}/{llama_inference_id}
-
Create a Mistral inference endpoint
Stack
8.15+
- PUT /_inference/{task_type}/{mistral_inference_id}
-
Create an Nvidia inference endpoint
Stack
9.3+
- PUT /_inference/{task_type}/{nvidia_inference_id}
-
Create an OpenAI inference endpoint
Stack
8.12+
- PUT /_inference/{task_type}/{openai_inference_id}
-
Create an OpenShift AI inference endpoint
Stack
9.3+
- PUT /_inference/{task_type}/{openshiftai_inference_id}
-
Create a VoyageAI inference endpoint
Stack
8.19+
- PUT /_inference/{task_type}/{voyageai_inference_id}
-
Create a Watsonx inference endpoint
Stack
8.16+
- PUT /_inference/{task_type}/{watsonx_inference_id}