What it is
Retrieval-augmented generation (RAG) gives a language model the documents it needs at the moment it answers. First a search step finds the passages most relevant to the question in your own collection; then those passages are placed in the prompt and the model writes its answer from them. Lewis and colleagues introduced the name and the method.
What it is for
Answering questions about material the model was not trained on or that changes: company documents, manuals, a course, recent news. It reduces made-up answers because the model can point to its passages, and it is usually cheaper and easier to update than fine-tuning.
How to do it
- Collect the documents and clean them: remove duplicates and page clutter, and keep their titles and dates.
- Split them into passages of a sensible length, a few paragraphs each, so that each holds one idea.
- Compute an embedding for each passage with an embedding model and store them in a search index or vector database.
- At question time, embed the question, fetch the closest few passages, and put them in the prompt with the instruction to answer only from them and to say when they do not contain the answer.
- Show the sources next to the answer, so a reader can check them.
- Test with real questions, including ones whose answer is not in the documents, and look at whether the right passages were retrieved: most failures come from retrieval, not from the model.