Look it up, then answer
RAG (retrieval-augmented generation) retrieves relevant documents or data before the model generates an answer, and has it respond from what was found. It produces an AI that consults sources rather than answering from memory, and it has become the default pattern in business deployments.
The two weaknesses it covers
LLMs do not know anything newer than their training data, and they will answer confidently about things they do not know. Retrieving current documents and handing them to the model addresses both at once.
To build an assistant that answers questions about your employee handbook, you do not retrain the model on the handbook. You search the handbook for the relevant section each time and let the model answer from it. That is the canonical use.
Roughly how it works
- Documents are split into small chunks, converted into numeric embeddings, and stored in a vector database
- When a question arrives, chunks with similar meaning are retrieved
- Those chunks go to the model with the question and an instruction to answer from them
Being able to attach a citation — section 3 of the relevant policy — is one of its most practical advantages.
You have already used it
Web search in ChatGPT, and tools that answer from documents you upload, work on the same principle. Compared with fine-tuning, updating the material is easy and the cost is far lower, which is why RAG is the first option considered when a model needs knowledge.
It has limits: if retrieval misses, the model cannot answer, and quality depends heavily on how documents were chunked. Even so, it remains the default starting point, and it appears in essentially every article about deploying AI in an organization.