RAG is not dead, it grew a brain
Agentic retrieval in practice: I let the model decide when and where to search, deleted half of my old pipeline, and finally understood which half mattered.
Twice this spring I read a confident post titled some variant of “RAG is dead”. Both times the body said something more modest: static retrieval pipelines are being replaced by retrieval that the model controls. That is not a death. That is a promotion.
A classic pipeline makes every decision upfront. Chunk size 500. Always fetch the five nearest pieces. Everything frozen before the first question arrives. An agentic one hands those decisions to the model at question time. It sees the question, decides whether to search at all, formulates its own queries, looks at what came back, and searches again with better terms if the first pass was thin. The pattern goes by agentic RAG, and the mechanics are unglamorous: retrieval is just another tool the agent calls in its loop.
What convinced me was not a benchmark, it was watching the traces. I gave both styles the same real question: why does a device reset under load. The fixed version fetched the five nearest chunks and produced a fluent answer about reset behavior in general. The agentic version searched, noticed the retrieved section mentioned an errata document, searched for the errata, found the actual known issue, and cited it. That second search is the whole story. A fixed pipeline has no way for finding something to lead to looking for something else. The follow-up question was the capability I had been missing without having a name for it.
The part that hurt a little: for our smaller document sets, the vector database turned out to be optional. There is a growing and slightly heretical school of thought that direct corpus access beats embedding indexes for a lot of real workloads: let the agent grep and read files like a person would, instead of pre-chewing everything into vectors. I tested it on a collection of roughly 2,000 wiki pages. Plain text search plus an agent that reads the hits beat the carefully tuned classic pipeline. Faster to build, easier to debug, and when it failed I could see why, because the search terms were readable text instead of a similarity score. I deleted the vector index for that collection and felt a strange mix of liberation and grief for my chunking code.
Embeddings kept their job where they are genuinely good: fuzzy matching across wording and language, the “find me things like this paragraph” cases, German umlauts meeting English questions. The honest summary of my current setup is boring: exact search where exact works, semantic search where meaning drifts, and an agent deciding which to reach for. Retrieval became a decision, not a setting.
One new discipline came with the freedom. When retrieval was fixed, its failures were at least predictable. An agent that decides for itself can decide badly, so the current best practice is to make it show its work: evaluate the evidence before generating, filter what does not support the answer, and keep the trace. Blindly pasting in whatever came back stopped being acceptable, and good riddance, because that is exactly how you get confident answers quoting the wrong table.
Cost me one evening of tuning to stop the agent from over-searching, though. Left alone it would happily do six searches for a question whose answer was in the first result, like a student padding an essay. A cap on tool calls and a “answer now if you can” nudge in the prompt fixed it. Agents inherit our vices along with our methods.