Agents still need retrieval because models can confidently use incorrect or outdated information supplied by the retrieval layer.
2
BM25 and dense vector search fail in different ways, so combining their rankings usually gives a better starting point than relying on either method alone.
3
Reranking can improve the ordering of a small candidate set, but its extra inference cost is justified only when the added precision matters.
Summary
Jeff Vestal and James Williams run a practical workshop on retrieval for agentic applications in Elasticsearch. They explain why dense vector search captures meaning but can miss exact identifiers, while BM25 handles exact terms but struggles with paraphrases. The workshop combines both methods with reciprocal rank fusion and linear score combination, then uses filtering, judgment sets, and search templates to improve and measure results. The speakers also show how Elastic Agent Builder connects retrieval with skills, tools, ES|QL, repeated searches, and citations that lead back to source documents. Their example agent diagnoses exit code 137 as an out-of-memory failure and retrieves settings that can prevent it. A final section covers reranking, including pointwise and listwise approaches. The main advice is practical: improve the evidence reaching the model, measure relevance changes, and add more inference only when its precision gain is worth the latency.
Agents remain dependent on the quality of retrieved information
Jeff Vestal says retrieval still matters because agents need information that may be time-sensitive, proprietary, or absent from model training. A model can be capable and still produce a bad answer when it receives incorrect documents and is told they are authoritative. The workshop therefore focuses on the retrieval layer rather than only on agent construction. The speakers use Elasticsearch to retrieve evidence, pass it to an agent, and inspect the sources behind the answer.
Chunking and embedding consistency affect semantic retrieval
Elasticsearch's semantic text field can chunk documents automatically. The speakers describe default sentence-boundary chunking into 250-token chunks with one sentence of overlap, while also allowing different boundaries and sizes. They stress that the same embedding model must be used when indexing documents and when processing queries. Embedding the inference configuration in the field helps avoid different developers using inconsistent models. Elasticsearch can generate embeddings through Elastic Inference Service, or developers can bring their own vectors and inference setup.
Dense vectors capture meaning but can miss exact identifiers
Dense vector search is useful when different questions express the same idea. The speakers use security configuration as an example of several differently worded questions that should retrieve similar documents. Semantic search becomes weaker when the query contains a very specific identifier, such as a product ID that has no meaningful language representation. BM25 and other lexical methods can recover those exact matches. The workshop treats the two approaches as complementary because each catches cases the other can miss.
Hybrid retrieval starts with combining rankings, then measuring the result
Reciprocal rank fusion runs lexical and semantic searches and combines their result positions rather than trying to compare raw scores. A document ranked highly by both searches is likely to rise in the combined list. Linear combination takes a different approach by normalizing scores to a common range and applying tunable weights. Judgment sets provide queries with documents that people consider correct, allowing teams to test different weights and compare relevance instead of tuning blindly.
Filtering reduces the candidate pool before ranking
The speakers describe filtering by fields such as keywords, location, or date as a way to narrow the candidate set before retrieval results are combined. A smaller pool gives the search methods fewer irrelevant documents to consider. Filtering can be used alongside hybrid search, and it can also be selected dynamically when the incoming query suggests a particular constraint. The workshop presents filtering as a simple technique that can improve retrieval without adding another model call.
Search templates can route different query types to different strategies
James Williams describes search templates as a way to keep several retrieval strategies available. One template might use reciprocal rank fusion, another linear combination, and another a combination followed by reranking. A layer above the templates can inspect the query and choose which one to invoke. The speakers describe this as a form of basic automatic tuning, although it still requires developers to define the strategies and the routing logic.
Agent Builder exposes retrieval, tools, workflows, and citations
Elastic Agent Builder connects agents to skills and tools that can search data, run workflows, call external APIs, and use MCP. Its APIs can also be reused in a custom application, so a prototype built in the interface does not have to be rewritten from scratch. In the workshop example, an agent investigates exit code 137, identifies the out-of-memory manager as the cause, and retrieves JVM settings that can prevent the failure. The agent can show the source documents and links behind its answer.
Reranking is useful when first-stage ordering is not precise enough
Reranking runs after an initial lexical, semantic, or hybrid search. The first stage retrieves a smaller candidate set, and a reranker scores the query and each candidate together. This adds inference at query time, so reranking the entire corpus would be expensive. Pointwise rerankers score documents independently. Listwise rerankers examine a group of candidates together, which can help when several documents are very similar. The speakers recommend adding reranking only when its precision gain fits the latency budget.
"If you feed these models, no matter how smart they are, garbage data and you confidently tell them this is the answer, pretty much they're going to believe you because that's what you're telling them."Jeff Vestal02:54
Who should watch
You are building a retrieval-augmented or agentic application and need to decide when to use BM25, vector search, hybrid search, or reranking.
Your search results look plausible but agents still answer incorrectly, and you need ways to inspect sources, create judgment sets, and test relevance changes.
You use Elasticsearch and want a hands-on path through semantic text, reciprocal rank fusion, linear score combination, search templates, Agent Builder, and ES|QL.