Create or update an ES|QL data source

PUT /_query/data_source/{name}

Auth

Creates or replaces a named, type-specific data source configuration for ES|QL data federation. Datasets reference data source configurations to access external data. Names must be lowercase and follow index or alias naming rules.

ES|QL Data Federation

Required privileges

Cluster privileges manage

Path parameters

name string required: The data source name to create or update.

Query parameters

master_timeout union oneOf · default: 30s: Period to wait for a connection to the master node.; Values: -1 0
timeout union oneOf · default: 30s: The time to wait for the request to be completed.; Values: -1 0

Request body

type string required: The data source type. Currently, s3 is supported. The value must be lowercase and contain no whitespace.
description string: A free-text description of the data source.
settings map string to object: Type-specific connection and authentication settings. For s3, connection settings include region and endpoint. Authentication settings include auth and the credentials required by the selected authentication method.

Response

application/json

acknowledged boolean required: For a successful response, this value is always true. On failure, an exception is returned instead.

View as Markdown

Examples

Request Console 1 / 8 languages

Create a data source for production logs stored in Amazon S3. The data source uses static credentials and connects to a bucket in the us-east-1 region.

Console Kibana Dev Tools

PUT /_query/data_source/prod_s3_logs
{
  "type": "s3",
  "description": "Production S3 logs bucket",
  "settings": {
    "region": "us-east-1",
    "auth": "static_credentials",
    "access_key": "<AWS_ACCESS_KEY_ID>",
    "secret_key": "<AWS_SECRET_ACCESS_KEY>"
  }
}

JavaScript Elasticsearch JavaScript Client

const response = await client.esql.putDataSource({
  name: "prod_s3_logs",
  type: "s3",
  description: "Production S3 logs bucket",
  settings: {
    region: "us-east-1",
    auth: "static_credentials",
    access_key: "<AWS_ACCESS_KEY_ID>",
    secret_key: "<AWS_SECRET_ACCESS_KEY>",
  },
});

Python Elasticsearch Python Client

resp = client.esql.put_data_source(
    name="prod_s3_logs",
    type="s3",
    description="Production S3 logs bucket",
    settings={
        "region": "us-east-1",
        "auth": "static_credentials",
        "access_key": "<AWS_ACCESS_KEY_ID>",
        "secret_key": "<AWS_SECRET_ACCESS_KEY>"
    },
)

curl Shell

curl -X PUT "$ELASTICSEARCH_URL/_query/data_source/prod_s3_logs" \
  -H "Authorization: ApiKey $ELASTIC_API_KEY" \
  -H "Content-Type: application/json" \
  -d '{
    "type": "s3",
    "description": "Production S3 logs bucket",
    "settings": {
      "region": "us-east-1",
      "auth": "static_credentials",
      "access_key": "\u003CAWS_ACCESS_KEY_ID\u003E",
      "secret_key": "\u003CAWS_SECRET_ACCESS_KEY\u003E"
    }
  }'

Java Elasticsearch Java Client

client.esql().putDataSource(p -> p
    .description("Production S3 logs bucket")
    .name("prod_s3_logs")
    .settings(Map.of("secret_key", JsonData.fromJson("\"<AWS_SECRET_ACCESS_KEY>\""),"auth", JsonData.fromJson("\"static_credentials\""),"access_key", JsonData.fromJson("\"<AWS_ACCESS_KEY_ID>\""),"region", JsonData.fromJson("\"us-east-1\"")))
    .type("s3")
);

C# Elasticsearch .NET Client

var response = await client.Esql
    .PutDataSourceAsync(name: "prod_s3_logs", d1 => d1
        .Description("Production S3 logs bucket")
        .Settings(new Dictionary<string, object>()
        {
            { "region", "us-east-1" },
            { "auth", "static_credentials" },
            { "access_key", "<AWS_ACCESS_KEY_ID>" },
            { "secret_key", "<AWS_SECRET_ACCESS_KEY>" }
        })
        .Type("s3")
    );

PHP Elasticsearch PHP Client

$resp = $client->esql()->putDataSource([
    "name" => "prod_s3_logs",
    "body" => [
        "type" => "s3",
        "description" => "Production S3 logs bucket",
        "settings" => [
            "region" => "us-east-1",
            "auth" => "static_credentials",
            "access_key" => "<AWS_ACCESS_KEY_ID>",
            "secret_key" => "<AWS_SECRET_ACCESS_KEY>",
        ],
    ],
]);

Ruby Elasticsearch Ruby Client

response = client.esql.put_data_source(
  name: "prod_s3_logs",
  body: {
    "type": "s3",
    "description": "Production S3 logs bucket",
    "settings": {
      "region": "us-east-1",
      "auth": "static_credentials",
      "access_key": "<AWS_ACCESS_KEY_ID>",
      "secret_key": "<AWS_SECRET_ACCESS_KEY>"
    }
  }
)

Response

The data source configuration was stored successfully.

{
  "acknowledged": true
}