Retrieval-augmented generation (RAG) combines retrieval of relevant documents with generative AI to produce accurate, context-aware answers grounded in retrieved information. The typical 2025 RAG pipeline flows as: query rewrite → hybrid retrieve (BM25 + dense) → rerank → top-k context → LLM with citations → answer, with optional HyDE, query expansion, and post-generation grounding checks. Several techniques refine the query side: query rewriting uses an LLM to transform a conversational or ambiguous query into a self-contained search query by resolving pronouns and expanding acronyms, dramatically improving recall on multi-turn dialogues; HyDE (Hypothetical Document Embeddings) prompts an LLM to generate a hypothetical answer first, embeds that, and retrieves documents similar to the hypothetical — useful for zero-shot domains where the original query is short; and step-back prompting generates a more abstract or general query alongside the original, retrieves context for both, and feeds both into the LLM, improving answers on factual, multi-hop, or scientific questions.
Multi-hop retrieval is required when a question demands chaining evidence across documents. Solutions include iterative retrieval (retrieve → answer → re-query), IRCoT (Interleaved Retrieval with Chain-of-Thought), and graph-based traversal over linked passages. Microsoft Research introduced GraphRAG in mid-2024, which builds a knowledge graph from documents (entities plus relations) and retrieves subgraphs relevant to the query, often using community detection to answer global questions like "what are the main themes?" — an approach that dramatically outperforms chunk-based RAG on such global sensemaking tasks. The next generation of RAG is agentic: agentic RAG gives the LLM tools and a loop to decide when and what to retrieve, when to re-query, when to call APIs, and when to answer — replacing the rigid retrieve-then-generate pipeline. Self-RAG trains the model to emit special [Retrieve] and [IsRel/IsSup/IsUse] reflection tokens so the LLM dynamically decides whether to retrieve and whether its own outputs are grounded and useful. CRAG (Corrective RAG) adds a lightweight retrieval evaluator that grades retrieved documents as Correct, Ambiguous, or Incorrect and triggers web search or fallback for low-confidence cases.
Four classic RAG failure modes are routinely diagnosed: missing content (not retrieved), missed top-ranked docs, not in context (lost in the middle), and not extracted (LLM ignores the answer even when present). The lost-in-the-middle problem refers to LLMs giving more weight to context at the beginning and end of the prompt and ignoring the middle, which is why rerankers should place the most relevant chunks at the top and bottom of the context window. A common default chunk size for long documents is 512 tokens with 10-20% overlap (~50-100 tokens); character/token chunking is fast but arbitrary, while semantic chunking splits at topic shifts using embedding similarity, producing more coherent chunks at the cost of latency. Two further research directions push beyond plain RAG: retrieval-augmented fine-tuning (RAFT) fine-tunes the LLM on QA pairs that include retrieved and sometimes irrelevant context, teaching it to ignore distractors, while RA-DIT (Retrieval-Augmented Dual Instruction Tuning, Meta 2023) separately fine-tunes the LLM and retriever on instruction data, improving both modules jointly.
The agentic pattern of fan-out — splitting a question into sub-questions, retrieving in parallel, and aggregating results — underlies Deep Research products and multi-hop agentic RAG. The historical roots of agentic retrieval reach back to ReAct (Reasoning + Acting), which prompts the LLM to interleave Thought → Action → Observation steps with actions like Search[query] and Lookup[link], and to Toolformer (Meta 2023), which self-supervises API calls by training only when the call actually improves perplexity. WebGPT (OpenAI 2021) showed that a GPT-3 model fine-tuned to browse the web and cite sources could produce answers preferred over human-written Reddit responses, foreshadowing today's AI search products.