Accelerate dense vector indexing with a GPU in Elasticsearch

GPU indexing in Elasticsearch builds Hierarchical Navigable Small World (HNSW) graphs with the NVIDIA cuVS library. Offloading that work to a graphics processing unit (GPU) speeds up dense vector ingest on large datasets and frees CPU resources for search and other tasks.

Before you begin

To use GPU indexing, you need:

An Enterprise subscription
A supported NVIDIA GPU (Ampere architecture or better, compute capability >= 8.0) with a minimum 8 GB of GPU memory
GPU driver, CUDA, and cuVS runtime libraries installed on the node. Refer to the Elastic support matrix for supported CUDA and cuVS versions.
LD_LIBRARY_PATH environment variable configured to include the cuVS libraries path and its dependencies (CUDA, rmm, and so on)
Supported platform: Linux x86_64 only, Java 22 or later
Supported dense vector configurations: hnsw and int8_hnsw; float element type only

Configuration

The vectors.indexing.use_gpu node-level setting controls GPU vector indexing.

Elasticsearch Docker image with GPU support

You can extend the official Elasticsearch Docker image with this example Dockerfile to add the dependencies required for GPU support.

Warning

This Dockerfile serves as an example implementation, and is not fully supported like our official Docker images.

This example is configured for Elasticsearch 9.5. To use a different Elasticsearch version, update both ELASTICSEARCH_VERSION and CUVS_VERSION to a supported combination from the Elastic support matrix.

Example Dockerfile

ARG ELASTICSEARCH_VERSION=9.5.2
FROM docker.elastic.co/elasticsearch/elasticsearch:${ELASTICSEARCH_VERSION}

USER root

# See https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/12.9.1/ubi9/base/Dockerfile?ref_type=heads
# and https://gitlab.com/nvidia/container-images/cuda/-/blob/master/dist/12.9.1/ubi9/devel/Dockerfile?ref_type=heads
# We are installing nvidia/cuda drivers/libraries the same way that nvidia does in their images

ENV CUVS_VERSION=26.04.00.194111

ENV NVARCH=x86_64
ENV NVIDIA_REQUIRE_CUDA="cuda>=12.9 brand=unknown,driver>=535,driver<536 brand=grid,driver>=535,driver<536 brand=tesla,driver>=535,driver<536 brand=nvidia,driver>=535,driver<536 brand=quadro,driver>=535,driver<536 brand=quadrortx,driver>=535,driver<536 brand=nvidiartx,driver>=535,driver<536 brand=vapps,driver>=535,driver<536 brand=vpc,driver>=535,driver<536 brand=vcs,driver>=535,driver<536 brand=vws,driver>=535,driver<536 brand=cloudgaming,driver>=535,driver<536 brand=unknown,driver>=550,driver<551 brand=grid,driver>=550,driver<551 brand=tesla,driver>=550,driver<551 brand=nvidia,driver>=550,driver<551 brand=quadro,driver>=550,driver<551 brand=quadrortx,driver>=550,driver<551 brand=nvidiartx,driver>=550,driver<551 brand=vapps,driver>=550,driver<551 brand=vpc,driver>=550,driver<551 brand=vcs,driver>=550,driver<551 brand=vws,driver>=550,driver<551 brand=cloudgaming,driver>=550,driver<551 brand=unknown,driver>=560,driver<561 brand=grid,driver>=560,driver<561 brand=tesla,driver>=560,driver<561 brand=nvidia,driver>=560,driver<561 brand=quadro,driver>=560,driver<561 brand=quadrortx,driver>=560,driver<561 brand=nvidiartx,driver>=560,driver<561 brand=vapps,driver>=560,driver<561 brand=vpc,driver>=560,driver<561 brand=vcs,driver>=560,driver<561 brand=vws,driver>=560,driver<561 brand=cloudgaming,driver>=560,driver<561 brand=unknown,driver>=565,driver<566 brand=grid,driver>=565,driver<566 brand=tesla,driver>=565,driver<566 brand=nvidia,driver>=565,driver<566 brand=quadro,driver>=565,driver<566 brand=quadrortx,driver>=565,driver<566 brand=nvidiartx,driver>=565,driver<566 brand=vapps,driver>=565,driver<566 brand=vpc,driver>=565,driver<566 brand=vcs,driver>=565,driver<566 brand=vws,driver>=565,driver<566 brand=cloudgaming,driver>=565,driver<566 brand=unknown,driver>=570,driver<571 brand=grid,driver>=570,driver<571 brand=tesla,driver>=570,driver<571 brand=nvidia,driver>=570,driver<571 brand=quadro,driver>=570,driver<571 brand=quadrortx,driver>=570,driver<571 brand=nvidiartx,driver>=570,driver<571 brand=vapps,driver>=570,driver<571 brand=vpc,driver>=570,driver<571 brand=vcs,driver>=570,driver<571 brand=vws,driver>=570,driver<571 brand=cloudgaming,driver>=570,driver<571"
ENV NV_CUDA_CUDART_VERSION=12.9.79-1
ENV CUDA_VERSION=12.9.1

ENV NV_CUDA_LIB_VERSION=12.9.1-1
ENV NV_NVPROF_VERSION=12.9.79-1
ENV NV_NVPROF_DEV_PACKAGE=cuda-nvprof-12-9-${NV_NVPROF_VERSION}
ENV NV_CUDA_CUDART_DEV_VERSION=12.9.79-1
ENV NV_NVML_DEV_VERSION=12.9.79-1
ENV NV_LIBCUBLAS_DEV_VERSION=12.9.1.4-1
ENV NV_LIBNPP_DEV_VERSION=12.4.1.87-1
ENV NV_LIBNPP_DEV_PACKAGE=libnpp-devel-12-9-${NV_LIBNPP_DEV_VERSION}
ENV NV_LIBNCCL_DEV_PACKAGE_NAME=libnccl-devel
ENV NV_LIBNCCL_DEV_PACKAGE_VERSION=2.27.3-1
ENV NCCL_VERSION=2.27.3
ENV NV_LIBNCCL_DEV_PACKAGE=${NV_LIBNCCL_DEV_PACKAGE_NAME}-${NV_LIBNCCL_DEV_PACKAGE_VERSION}+cuda12.9
ENV NV_CUDA_NSIGHT_COMPUTE_VERSION=12.9.1-1
ENV NV_CUDA_NSIGHT_COMPUTE_DEV_PACKAGE=cuda-nsight-compute-12-9-${NV_CUDA_NSIGHT_COMPUTE_VERSION}

ENV NV_NVTX_VERSION=12.9.79-1
ENV NV_LIBNPP_VERSION=12.4.1.87-1
ENV NV_LIBNPP_PACKAGE=libnpp-12-9-${NV_LIBNPP_VERSION}
ENV NV_LIBCUBLAS_VERSION=12.9.1.4-1
ENV NV_LIBNCCL_PACKAGE_NAME=libnccl
ENV NV_LIBNCCL_PACKAGE_VERSION=2.27.3-1
ENV NV_LIBNCCL_VERSION=2.27.3
ENV NCCL_VERSION=2.27.3
ENV NV_LIBNCCL_PACKAGE=${NV_LIBNCCL_PACKAGE_NAME}-${NV_LIBNCCL_PACKAGE_VERSION}+cuda12.9

ENV NVIDIA_VISIBLE_DEVICES=all
ENV NVIDIA_DRIVER_CAPABILITIES=compute,utility
ENV RAFT_DEBUG_LOG_FILE=/dev/null

# Install nvidia drivers
RUN microdnf install -y dnf
RUN dnf install -y 'dnf-command(config-manager)'
RUN dnf config-manager --add-repo https://developer.download.nvidia.com/compute/cuda/repos/rhel9/x86_64/cuda-rhel9.repo

RUN dnf upgrade -y && dnf install -y \
    cuda-cudart-12-9-${NV_CUDA_CUDART_VERSION} \
    cuda-compat-12-9 \
    && dnf clean all \
    && rm -rf /var/cache/yum/*

# Set up env vars for various libraries (cuda, libcuvs)
RUN echo "/usr/local/cuda/lib64" >> /etc/ld.so.conf.d/nvidia.conf
ENV PATH=/usr/local/nvidia/bin:/usr/local/cuda/bin:${PATH}
ENV LIBCUVS_DIR="/opt/cuvs"
ENV LIBCUVS_ARCHIVE_DIR="libcuvs-linux-x86_64-${CUVS_VERSION}_cuda12-archive"
ENV LD_LIBRARY_PATH=${LIBCUVS_DIR}/${LIBCUVS_ARCHIVE_DIR}/lib:/usr/local/nvidia/lib:/usr/local/nvidia/lib64:/usr/local/cuda/lib64

# Install other required nvidia and cuda libraries, as well as tar and xz
RUN dnf install -y \
    cuda-libraries-12-9-${NV_CUDA_LIB_VERSION} \
    cuda-nvtx-12-9-${NV_NVTX_VERSION} \
    ${NV_LIBNPP_PACKAGE} \
    libcublas-12-9-${NV_LIBCUBLAS_VERSION} \
    ${NV_LIBNCCL_PACKAGE} \
    tar xz \
    && dnf clean all \
    && rm -rf /var/cache/yum/*

# Install the libcuvs runtime from NVIDIA's distribution archive
RUN mkdir -p "$LIBCUVS_DIR" && \
    chmod 775 "$LIBCUVS_DIR" && \
    cd "$LIBCUVS_DIR" && \
    CUVS_ARCHIVE="${LIBCUVS_ARCHIVE_DIR}.tar.xz" && \
    curl -fO "https://developer.download.nvidia.com/compute/cuvs/redist/libcuvs/linux-x86_64/$CUVS_ARCHIVE" && \
    tar -xJf "$CUVS_ARCHIVE" && \
    rm -f "$CUVS_ARCHIVE"

# Reset the user back to elasticsearch
USER 1000:0

Host requirements

The host machine running the Docker container needs NVIDIA Container Toolkit installed and configured.

Build it

docker build -t es-gpu .

Run it

docker run \
  -p 9200:9200 \
  -p 9300:9300 \
  -e "discovery.type=single-node" \
  -e "xpack.security.enabled=false" \
  -e "xpack.license.self_generated.type=trial" \
  -e "vectors.indexing.use_gpu=true" \
  --user elasticsearch \
  --gpus all \
  --rm -it es-gpu

Monitoring

Use the GET _xpack/usage API to monitor GPU vector indexing status and usage across all nodes in the cluster:

GET _xpack/usage?filter_path=gpu_vector_indexing

{
  "gpu_vector_indexing": {
    "available": true,
    "enabled": true,
    "index_build_count": 30,
    "nodes_with_gpu": 3,
    "nodes": [
      { "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
        "enabled": true, "index_build_count": 10 },
      { "type": "NVIDIA L4", "memory_in_bytes": 24000000000,
        "enabled": true, "index_build_count": 10 },
      { "type": "NVIDIA A100", "memory_in_bytes": 80000000000,
        "enabled": true, "index_build_count": 10 }
    ]
  }
}

Whether the current license permits GPU indexing.
Whether at least one node has GPU hardware configured and has not turned it off through vectors.indexing.use_gpu=false.
Total number of GPU index builds across the cluster.
Number of data nodes with GPU support.
Per-node GPU details including type, memory, enabled status, and build count.

Troubleshooting

By default, Elasticsearch uses GPU indexing for supported vector types if a compatible GPU and required libraries are detected. Check server logs for messages indicating whether Elasticsearch has detected a GPU.

If the following message appears, a GPU was successfully detected and GPU indexing is used:

[o.e.x.g.GPUSupport ] [elasticsearch-0] Found compatible GPU [NVIDIA L4] (id: [0])

If this message doesn't appear, check for warning messages explaining why GPU indexing isn't being used, such as an unsupported environment, missing libraries, or an incompatible GPU.

cuVS runtime version mismatch

If the cuVS runtime library is older than the cuvs-java library bundled with Elasticsearch, you see a warning similar to the following:

GPU based vector indexing is not supported on this platform; Cannot create JDKProvider:
Version mismatch: outdated libcuvs_c (libcuvs_c [25.12.0], cuvs-java version [26.02.0])

Install a cuVS runtime version that's supported for your Elasticsearch version. Refer to the Elastic support matrix.

Node fails to start with `vectors.indexing.use_gpu: true`

To enforce GPU indexing, set vectors.indexing.use_gpu: true in elasticsearch.yml. The node fails to start if GPU indexing isn't available. For example, if a GPU isn't detected by Elasticsearch, if the runtime isn't supported, or if the necessary dependencies aren't correctly configured.

If the node fails to start, check:

A supported NVIDIA GPU is present
CUDA runtime libraries and drivers are installed (check with nvidia-smi)
LD_LIBRARY_PATH includes paths to the cuVS libraries and to their dependencies (for example, CUDA)
Supported platform: Linux x86_64 with Java 22 or later

Performance not improved with GPU indexing

If you're sure that GPU indexing is enabled but performance doesn't improve, check the following:

Use supported vector index types and the float element type.
Use a dataset large enough to benefit from GPU acceleration.
Check for other bottlenecks. GPU indexing accelerates HNSW graph building, but other factors can limit speedups.
- Indexing throughput depends on how fast you can get data into Elasticsearch. Check network speed and client performance. Use multiple clients if needed.
- JSON parsing can dominate the computation. Use base64 encoded vectors instead of JSON arrays.
- Storage speed also matters. The GPU can process lots of data, so use storage that can keep up. Avoid network-attached storage, and prefer fast NVMe.
Monitor CPU usage to confirm work is offloaded to the GPU.
Monitor GPU usage (for example, with nvidia-smi).

Next steps

Index dense vectors and run kNN search
Tune approximate kNN search for indexing and query performance
Bring your own dense vectors if you already have embeddings