kNN retriever

A kNN retriever returns top documents from a k-nearest neighbor search (kNN).

field

(Required, string)

The name of the vector field to search against. Must be a dense_vector field with indexing enabled. To run a kNN search on a semantic_text field, use the knn query instead.

query_vector

(Required if query_vector_builder is not defined, array of float or string)

Query vector. Must have the same number of dimensions as the vector field you are searching against. Must be one of:

• An array of floats
• A hex-encoded byte vector (one byte per dimension; for bit, one byte per 8 dimensions). Stack 9.0-9.3 The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available from 9.0 to 9.3 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
• A base64-encoded vector string. Base64 supports float and bfloat16 (big-endian), byte, and bit encodings depending on the target field type. Stack 9.4+ The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.4 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. Serverless Elastic Cloud Serverless projects are autoscaled environments, fully managed by Elastic and available on Elastic Cloud. Availability Generally available This functionality is generally available and ready for production usage. Serverless interfaces and procedures might differ from classic Elastic Stack deployments.

query_vector_builder

(Required if query_vector is not defined, query vector builder object)

Defines a model to build a query vector.

k

(Required, integer)

Number of nearest neighbors to return as top hits. This value must be fewer than or equal to num_candidates.

num_candidates

(Optional, integer Stack 9.5+ The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.5 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes. ; in earlier versions this parameter is required)

The number of nearest neighbor candidates to consider per shard. Needs to be greater than k, or size if k is omitted, and cannot exceed 10,000. Elasticsearch collects num_candidates results from each shard, then merges them to find the top k results. Increasing num_candidates tends to improve the accuracy of the final k results. Defaults to Math.min(1.5 * k, 10_000).

visit_percentage Stack 9.2+ The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.2 This functionality is generally available and ready for production usage. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

(Optional, float)

The percentage of vectors to explore per shard while doing knn search with bbq_disk. Must be between 0 and 100. 0 will default to using num_candidates for calculating the percent visited. Increasing visit_percentage tends to improve the accuracy of the final results. If visit_percentage is set for bbq_disk, num_candidates is ignored. Defaults to ~1% per shard for every 1 million vectors.

filter

(Optional, query object or list of query objects)

Query to filter the documents that can match. The kNN search will return the top k documents that also match this filter. The value can be a single query or a list of queries. If filter is not provided, all documents are allowed to match.

similarity

(Optional, float)

The minimum similarity required for a document to be considered a match. The similarity value calculated relates to the raw similarity used. Not the document score. The matched documents are then scored according to similarity and the provided boost is applied.

The similarity parameter is the direct vector similarity calculation.

• l2_norm: also known as Euclidean, will include documents where the vector is within the dims dimensional hypersphere with radius similarity with origin at query_vector.
• cosine, dot_product, and max_inner_product: Only return vectors where the cosine similarity or dot-product are at least the provided similarity.

Read more here: knn similarity search

rescore_vector Stack GA 9.1+ … The Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats. Availability Generally available since 9.1 This functionality is generally available and ready for production usage. Preview in 9.0 This functionality is in technical preview and is ready for evaluation. Use with caution in production; it is not recommended for mission-critical workloads. Elastic will work to fix any issues, but features in technical preview are not subject to the support SLA of official GA features. Specific Support terms apply. Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments. This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.

(Optional, object) Apply oversampling and rescoring to quantized vectors.

oversample

(Required, float)

Applies the specified oversample factor to k on the approximate kNN search. The approximate kNN search will:

• Retrieve num_candidates candidates per shard.
• From these candidates, the top k * oversample candidates per shard will be rescored using the original vectors.
• The top k rescored candidates will be returned.

See oversampling and rescoring quantized vectors for details.

The parameters query_vector and query_vector_builder cannot be used together.

GET /restaurants/_search
{
  "retriever": {
    "knn": {
      "field": "vector",
      "query_vector": [10, 22, 77],
      "k": 10,
      "num_candidates": 10
    }
  }
}

1. Configuration for k-nearest neighbor (knn) search, which is based on vector similarity.
2. Specifies the field name that contains the vectors.
3. The query vector against which document vectors are compared in the knn search.
4. The number of nearest neighbors to return as top hits. This value must be fewer than or equal to num_candidates.
5. The size of the initial candidate set from which the final k nearest neighbors are selected.