Retrieval-augmented generation (RAG) pairs a retriever that fetches relevant external documents with a generator LLM that conditions its answer on those documents. RAG reduces hallucinations, supplies up-to-date knowledge beyond the model's training cutoff, enables private knowledge bases, and grounds answers in citable sources. Because language models only know what was in their training data, queries about events after the knowledge cutoff are a common source of fabrication unless augmented by search or RAG.
The retriever depends on embeddings, which are dense vector representations of text that capture semantic meaning so that similar phrases map to nearby points. In semantic search, a query is embedded and compared to pre-embedded documents via cosine or dot-product similarity. Hybrid search combines dense vector similarity with sparse lexical retrieval like BM25, often blending scores through reciprocal rank fusion to capture both conceptual meaning and exact keyword matches. Re-ranking applies a more powerful cross-encoder in a second retrieval pass to score and reorder the initial top-k candidates, improving precision before chunks reach the generator.
Because embeddings and generators have finite context windows, large documents must be split into smaller passages through chunking. Semantic chunking groups text by meaning—paragraphs, sections, or embedding-based boundaries—so each chunk is self-contained and retrieves more coherently than fixed-size chunking. Query rewriting transforms the user's raw question by expanding acronyms, adding synonyms, or decomposing multi-part questions to improve recall. Self-RAG goes further by letting the model decide when to retrieve, what to retrieve, and how to critique its own answers using reflection tokens. RAG quality is typically measured by groundedness—whether the answer is supported by retrieved context—distinct from general factual accuracy.