Standard RAG runs one fixed pass, embed, fetch the nearest matches, generate, regardless of whether those matches actually contain the answer. That works for narrow, well-indexed knowledge bases, and breaks on ambiguous questions that span multiple documents. A 2025 arXiv survey on the pattern describes agentic RAG as embedding autonomous agents directly into the retrieval pipeline, using reflection, planning, and tool use to manage retrieval strategy dynamically rather than following one fixed sequence (Singh et al., arXiv:2501.09136).
Standard RAG retrieves once and hopes. Agentic RAG checks its own retrieval before it lets the model answer from it.
How does the retrieve-evaluate-retry loop work?
Four steps repeat until the agent is satisfied or hits a retry limit: formulate a query, retrieve candidates, evaluate them against the question, then either generate, reformulate and retry, or escalate to a different source. This is the same plan, act, observe cycle that defines agentic AI, applied specifically to retrieval.
What architecture patterns implement this?
Corrective RAG adds a relevance check and falls back to a different source when retrieved documents score poorly. Self-RAG decides mid-generation whether it needs more evidence. Router-based RAG classifies the question first: a financial question routes to a database, a policy question to a document store, each reached through a consistent interface such as an MCP server. All three trade a cheap fixed pass for a variable number of passes that only runs as long as the question requires.
When does the added cost actually earn its keep?
It earns its cost on multi-hop questions, knowledge spread across systems with no single index, or domains where a confident wrong answer is expensive. It does not earn its cost on a narrow FAQ where one pass already answers correctly most of the time. The judgment call is the same one that governs AI agent design generally: match complexity to the task’s actual variance.
What can go wrong, and how do you catch it?
The evaluation step is itself a model’s judgment call, and a model wrong about whether it has enough evidence will retry when it should stop, or stop when it should retry, invisibly, the same failure mode covered in why self-reported confidence is not a reliable signal. Cap retries with a hard limit and a visible cost and latency budget, so a runaway loop becomes an alert instead of a silent bill, and log every retrieval attempt so a wrong answer traces back to a specific step rather than an unexplainable model error.