Optimize model context

Warning

Playground was deprecated in Elastic Stack 9.4 and removed in 9.6. It is unavailable in Elastic Serverless. Use Agent Builder (generally available since 9.3) instead.

With Agent Builder you can:

Interact with your Elasticsearch data using a chat interface in full-screen and sidebar modes
Use built-in agents, tools, and skills out of the box
Create custom agents, custom tools, and custom skills
Integrate via REST API and MCP
Combine LLM reasoning with Elastic Workflows to automate complex processes

Context is the information you provide to the LLM, to optimize the relevance of your query results. Without additional context, an LLM generates results solely based on its training data. In Playground, this additional context is the information contained in your Elasticsearch indices.

There are a few ways to optimize this context for better results. Some adjustments can be made directly in the Playground UI. Others require refining your indexing strategy, and potentially reindexing your data.

Note

Only one field can be selected as context for the LLM.

Edit context in UI

Use the Playground context section in the Playground UI to adjust the number of documents and fields sent to the LLM.

If you’re hitting context length limits, try the following:

Limit the number of documents retrieved
Pick a field with less tokens, reducing the context length

Other context optimizations

This section covers additional context optimizations that you won’t be able to make directly in the UI.

Chunking large documents

If you're working with large fields, you might need to adjust your indexing strategy. Consider breaking your documents into smaller chunks, such as sentences or paragraphs.

If you don’t yet have a chunking strategy, start by chunking your documents into passages.

Otherwise, consider updating your chunking strategy, for example, from sentence based to paragraph based chunking.

Refer to the following Python notebooks for examples of how to chunk your documents:

Optimizing context for cost and performance

The following recommendations can help you balance cost, latency, and result quality when working with different context sizes:

Optimize context length

Determine the optimal context length through empirical testing. Start with a baseline and adjust incrementally to find a balance that optimizes both response quality and system performance.

Implement token pruning for ELSER model

If you’re using our ELSER model, consider implementing token pruning to reduce the number of tokens sent to the model. Refer to these relevant blog posts:

Monitor and adjust

Continuously monitor the effects of context size changes on performance and adjust as necessary.