AuthThe Elastic Stack includes Elastic's core products such as Elasticsearch, Kibana, Logstash, and Beats.AvailabilityExperimentalThis functionality is experimental and is not ready for production usage. Experimental features may change or be removed at any time. Elastic will work to fix any issues, but experimental features are not subject to the support SLA of official GA features. Specific Support terms apply.Unless stated otherwise on the page, this functionality is available when your Elastic Stack is deployed on Elastic Cloud Hosted, Elastic Cloud Enterprise, Elastic Cloud on Kubernetes, and self-managed environments.This documentation corresponds to the latest patch available for each minor version. If you're not using the latest patch, check the release notes for changes.
Estimate the recall and cost of approximate kNN search configurations on a dense_vector field against a more thorough baseline.
The field must use the bbq_disk index type, and must not be inside a nested object.
Each knn_settings entry reports the mean recall of the baseline's top k, not of relevance judgments, plus included and excluded query counts and cost totals.
A query is excluded from the mean recall when the candidate finds a document scoring above the baseline's lowest top k score.
A baseline returning fewer than k hits fails that query, because recall at k is undefined.
All configurations run against one point in time.
The default baseline uses a visit_percentage of 20 and a rescore_vector.oversample of 100.
An exact full-precision baseline must be requested explicitly with allow_exact_baseline, and is limited to 100 million vector-by-query comparisons.
The request is refused when the search.knn_eval.enabled cluster setting or the index.knn_eval.enabled index setting is false.
If false, the request returns an error if it targets a missing or closed index.
allow_no_indicesboolean · default: true
If false, the request returns an error if any wildcard expression, index alias, or _all value targets only missing or closed indices.
This behavior applies even if the request targets other open indices.
expand_wildcardsuniononeOf · default: open
The type of index that wildcard patterns can match.
It supports comma-separated values, such as open,hidden.
Supported values include:
all: Match any data stream or index, including hidden ones.
open: Match open, non-hidden indices. Also matches any non-hidden data stream.
closed: Match closed, non-hidden indices. Also matches any non-hidden data stream. Data streams cannot be closed.
hidden: Match hidden data streams and hidden indices. Must be combined with open, closed, or both.
none: Wildcard expressions are not accepted.
Values:allopenclosedhiddennone
wait_for_completionboolean · default: true
If true, the request waits for the evaluation to complete.
A large evaluation can run for minutes, and closing the connection, for example because of a client or proxy timeout, cancels it.
If false, the request returns a task identifier, and stores the completed response for retrieval through the task management API.
The number of nearest neighbors to compute the recall for, from 1 to 1000.
query_sourceuniononeOfrequired
The source of the query vectors.
One of:(determined by from)
from: docs
Samples the query vectors from the stored documents.
fromenumrequired
Values:docs
sizenumberrequired
The number of documents to sample, from 1 to 1000.
seednumber
The seed of the sampling, to sample the same documents across requests.
from: vectors
Uses the query vectors in the request.
fromenumrequired
Values:vectors
vectorsobject[]required
The query vectors, from 1 to 1000 entries with unique identifiers.
idstringrequired
The identifier of the query, which identifies it in the failures of the response.
query_vectoruniononeOfrequired
The query vector.
One of:number[]orstring
baselineobject
The settings of the search that the knn_settings entries are compared with.
If not specified, it uses a visit_percentage of 20 and a rescore_vector.oversample of 100.
visit_percentagenumber
The percentage of postings to visit, from 0 to 100.
0 selects the automatic visit calculation.
num_candidatesnumber
The number of nearest neighbor candidates to consider per shard.
It can't be less than k, or more than 10000 (9999 with sampling).
rescore_vectorobject
The rescoring of the candidates. Only applies to quantized fields.
oversamplenumberrequired
Applies the specified oversample factor to k on the approximate kNN search
exactboolean · default: false
If true, compares every vector at full precision.
Only applies to the baseline, requires allow_exact_baseline, and can't be combined with other settings.
knn_settingsobject[]required
The kNN search settings to evaluate, from 1 to 32 unique entries.
visit_percentagenumber
The percentage of postings to visit, from 0 to 100.
0 selects the automatic visit calculation.
num_candidatesnumber
The number of nearest neighbor candidates to consider per shard.
It can't be less than k, or more than 10000 (9999 with sampling).
rescore_vectorobject
The rescoring of the candidates. Only applies to quantized fields.
oversamplenumberrequired
Applies the specified oversample factor to k on the approximate kNN search
exactboolean · default: false
If true, compares every vector at full precision.
Only applies to the baseline, requires allow_exact_baseline, and can't be combined with other settings.
allow_exact_baselineboolean · default: false
If true, the baseline can be exact.
An exact baseline compares every vector at full precision, and can severely slow down other searches.
The source of the query vectors.
Not returned when wait_for_completion is false.
fromenumrequired
How the query vectors were chosen.
Values:docsvectors
baselineobject
The settings of the baseline search.
Not returned when wait_for_completion is false.
visit_percentagenumber
The percentage of postings to visit, from 0 to 100.
0 selects the automatic visit calculation.
num_candidatesnumber
The number of nearest neighbor candidates to consider per shard.
It can't be less than k, or more than 10000 (9999 with sampling).
rescore_vectorobject
The rescoring of the candidates. Only applies to quantized fields.
oversamplenumberrequired
Applies the specified oversample factor to k on the approximate kNN search
exactboolean · default: false
If true, compares every vector at full precision.
Only applies to the baseline, requires allow_exact_baseline, and can't be combined with other settings.
rescore_window_cappedboolean · default: false
If true, the number of vectors to rescore exceeded the limit of 10000.
quantized_scoresboolean · default: false
If true, the hits were not rescored, so their scores are estimates, and no query is excluded from the mean recall.
baseline_took_msnumber
The time the baseline searches took, in milliseconds.
Not returned when wait_for_completion is false.
baseline_vector_opsnumber
The number of vector operations of the baseline searches.
Not returned when wait_for_completion is false.
baseline_vector_ops_kindenum
What baseline_vector_ops counts.
Not returned when wait_for_completion is false.
Supported values include:
full_precision_scan: Full-precision comparisons across all the matching vectors.
quantized_visit_plus_rescore: Quantized visits plus full-precision rescoring.
The results of each knn_settings entry, in the order of the request.
Not returned when wait_for_completion is false.
knn_settingsobjectrequired
The evaluated settings.
visit_percentagenumber
The percentage of postings to visit, from 0 to 100.
0 selects the automatic visit calculation.
num_candidatesnumber
The number of nearest neighbor candidates to consider per shard.
It can't be less than k, or more than 10000 (9999 with sampling).
rescore_vectorobject
The rescoring of the candidates. Only applies to quantized fields.
oversamplenumberrequired
Applies the specified oversample factor to k on the approximate kNN search
exactboolean · default: false
If true, compares every vector at full precision.
Only applies to the baseline, requires allow_exact_baseline, and can't be combined with other settings.
rescore_window_cappedboolean · default: false
If true, the number of vectors to rescore exceeded the limit of 10000.
quantized_scoresboolean · default: false
If true, the hits were not rescored, so their scores are estimates, and no query is excluded from the mean recall.
recallunionnumber | stringrequirednullable
The mean recall of the included queries, or null if no query is included.
included_queriesnumberrequired
The number of queries included in the mean recall.
excluded_queriesnumberrequired
The number of queries excluded from the mean recall, because the search found a document scoring above the baseline's lowest top k score.
took_msnumberrequired
The time the searches took, in milliseconds.
vector_opsnumberrequired
The number of vector operations of the searches.
failuresmapstring to object
The queries that failed, by query identifier.
Not returned when wait_for_completion is false.
<string>object
errorobjectrequired
typestringrequired
The type of error
reasonunionstringnullable
A human-readable explanation of the error, in English.
One of:string
stack_tracestring
The server stack trace. Present only if the error_trace=true parameter was sent with the request.