Operations
Retrieval-augmented generation (RAG)
Also called: RAG, grounded generation
Definition
Retrieval-augmented generation is the pattern of fetching the relevant source documents first and asking the model to answer only from them, so the answer is traceable to evidence.
RAG shifts the hard problem from the model to the retrieval layer. If the right paragraph reaches the model, a competent model will usually answer well; if it does not, no amount of prompting saves the answer, because the model can only work with what it was given. In practice most RAG failures are search failures — bad chunking, missing documents, stale indexes — rather than reasoning failures, which is a diagnosis most teams get wrong at first.
The retrieval layer is the product
Document quality, chunk boundaries, permissions, and freshness determine the ceiling of the whole system, regardless of which model sits on top of it. Organisations that already curate their policy wordings, loss runs, and procedures start several quarters ahead of those that do not, because the retrieval layer they need to build is largely a matter of exposing organized data that already exists, rather than first cleaning up years of inconsistent, duplicated, or poorly indexed documents before retrieval can even begin to work reliably.
Citations are the control
Requiring the system to show which passage supported each statement makes the output auditable, gives reviewers a fast way to check it, and turns a black box into something a risk committee can sign off on with confidence. A reviewer who can click through to the source passage in seconds is far more likely to catch an error than one asked to independently verify an unsupported claim from scratch, which is the practical difference citations make in day-to-day review workload, not just in theoretical auditability.
Freshness and permissions are the recurring failure modes
A RAG system that answers confidently from a document that was superseded six months ago is arguably worse than one with no retrieval at all, because the citation gives the wrong answer false credibility. Keeping the index current, and enforcing who is allowed to see which documents at the moment of retrieval rather than only at the point of upload, are the two operational disciplines that determine whether a RAG system stays trustworthy after launch instead of quietly degrading as the underlying document set changes.
In practice
- Measure retrieval quality on its own before blaming the model
- Enforce source permissions at retrieval time, not in the prompt
- Show citations in the interface so reviewers can verify in seconds
- Monitor document freshness and retire superseded sources from the index promptly
