Type to search.

Perform chat completion inference

Auth The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 8.18 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

The chat completion inference API enables real-time responses for chat completion tasks by delivering answers incrementally, reducing response times during computation. It only works with the chat_completion task type.

Note

The chat_completion task type is only available within the _stream API and only supports streaming. The Chat completion inference API and the Stream inference API differ in their response structure and capabilities. The Chat completion inference API provides more comprehensive customization options through more fields and function calling support. To determine whether a given inference service supports this task type, please see the page for that service.

Path parameters

inference_id string required

The inference Id

Query parameters

timeout union oneOf · default: 30s

Specifies the amount of time to wait for the inference request to complete.

Values: -1 0

Request body

messages object[] required

A list of objects representing the conversation. Requests should generally only add new messages from the user (role user). The other message roles (assistant, system, or tool) should generally only be copied from the response to a previous completion request, such that the messages array is built up throughout a conversation.

content union oneOf

The content of the message.

String example:

Text 
{
   "content": "Some string"
}

Object example:

Text 
{
  "content": [
      {
       "text": "Some text",
       "type": "text"
      }
   ]
}
One of:
role string required

The role of the message author. Valid values are user, assistant, system, and tool.

tool_call_id string

Only for tool role messages. The tool call that this message is responding to.

model string

The ID of the model to use.

max_completion_tokens number

The upper bound limit for the number of tokens that can be generated for a completion request.

stop string[]

A sequence of strings to control when the model should stop generating additional tokens.

temperature number

The sampling temperature to use.

tool_choice union oneOf

Controls which tool is called by the model. String representation: One of auto, none, or requrired. auto allows the model to choose between calling tools and generating a message. none causes the model to not call any tools. required forces the model to call one or more tools. Example (object representation):

Text 
{
  "tool_choice": {
      "type": "function",
      "function": {
          "name": "get_current_weather"
      }
  }
}
One of:
top_p number

Nucleus sampling, an alternative to sampling with temperature.

Response

application/json

Response Type: object