Type to search.

Create an Amazon Bedrock inference endpoint

Auth

Create an inference endpoint to perform an inference task with the amazonbedrock service.

Note

You need to provide the access and secret keys only once, during the inference model creation. The get inference API does not retrieve your access or secret keys. After creating the inference model, you cannot change the associated key pairs. If you want to use a different access and secret key pair, delete the inference model and recreate it with the same name and the updated keys.

Required privileges

Cluster privileges manage_inference

Path parameters

task_type enum required

The type of the inference task that the model will perform.

Values: chat_completion completion text_embedding
amazonbedrock_inference_id string required

The unique identifier of the inference endpoint.

Query parameters

timeout union oneOf · default: 30s

Specifies the amount of time to wait for the inference endpoint to be created.

Values: -1 0

Request body

service enum required

The type of service supported for the specified task type. In this case, amazonbedrock.

Values: amazonbedrock
service_settings object required

Settings used to install the inference model. These settings are specific to the amazonbedrock service.

access_key string required

A valid AWS access key that has permissions to use Amazon Bedrock and access to models for inference requests.

model string required

The base model ID or an ARN to a custom model based on a foundational model. The base model IDs can be found in the Amazon Bedrock documentation. Note that the model ID must be available for the provider chosen and your IAM user must have access to the model.

External docs
provider string

The model provider for your deployment. Note that some providers may support only certain task types. Supported providers include:

  • amazontitan - available for text_embedding and completion task types
  • anthropic - available for chat_completion and completion task types
  • ai21labs - available for chat_completion and completion task types
  • cohere - available for chat_completion, completion and text_embedding task types
  • meta - available for chat_completion and completion task types
  • mistral - available for chat_completion and completion task types
region string required

The region that your model or ARN is deployed in. The list of available regions per model can be found in the Amazon Bedrock documentation.

External docs
secret_key string required

A valid AWS secret key that is paired with the access_key. For informationg about creating and managing access and secret keys, refer to the AWS documentation.

External docs

Response

application/json
service string required

The service type

service_settings object required

Settings specific to the service

task_settings object

Task settings specific to the service and task type

inference_id string required

The inference Id

task_type enum required

The task type

Values: chat_completion completion text_embedding