AI Search Results

This deck brings together a wide range of questions about the fast-moving world of AI-powered search. It covers the basics of how AI search differs from traditional keyword-based search, dives into technical concepts like neural information retrieval and speech-to-retrieval engines, and looks at market trends, forecasts, and regional developments through mid-2025. You'll also find questions tied to broader AI themes, such as the role of information retrieval in artificial general intelligence and how generative AI is reshaping search results.

It's a great fit for anyone who wants to stay current with where search technology is heading, whether you're a student, a product or marketing professional, a developer, or simply a curious learner tracking the AI space. Because the questions mix factual data points with conceptual explanations, the deck works well as both a knowledge check and a way to surface gaps in your understanding of the topic.

To get the most out of these cards, try reviewing them in short, spaced sessions rather than cramming all at once — many of the answers involve specific numbers and named technologies that stick better with repeated exposure over several days. It can also help to group related cards mentally as you go, for example clustering the market and forecast questions together and the technical definitions in another batch, so the connections between them start to feel natural rather than isolated facts.

AI search has moved from research curiosity to commercial reality in the mid-2020s. By June 2025, AI-driven search traffic accounts for roughly 7.82% of the combined search market — a significant but still emerging share compared to established engines like Google. Forecasts extending from 2025 to 2030 project that ChatGPT and other AI-powered search interfaces will continue gaining ground, with some analysts predicting they may eventually rival or surpass traditional keyword-based search engines.

The shift from traditional to AI search is more than a cosmetic change. Traditional engines rely on keyword matching and link-based ranking signals like PageRank, whereas AI search uses semantic understanding and machine learning to interpret both queries and documents. Modern AI search engines deliver faster, smarter, and more accurate results by combining machine learning, natural language processing, and context understanding. Generative AI in particular transforms search outcomes by understanding specific inputs such as Boolean terms, connectors, and filters, providing more contextually relevant answers than keyword systems ever could.

Underpinning this transition is Neural Information Retrieval (Neural IR), an AI approach in which search engines use neural networks to understand and retrieve information more efficiently. Neural IR is widely considered a cornerstone of modern AI search, and broader Information Retrieval research is in turn a cornerstone of AI development itself, enabling systems to find, access, and process relevant data from vast datasets. The major technologies powering this revolution include neural networks, machine learning, natural language processing, vector embeddings, and transformer models.

Regional and product dynamics are also evolving. China's AI search market, according to Omdia's 2025 analysis, is integrating generative AI directly with conventional internet search to create hybrid solutions tailored to regional needs. Innovations like Google's 2025 Speech-to-Retrieval engine — which interprets voice queries and retrieves information without intermediate text conversion — show how the user experience is being rethought from the ground up. Yet the technology still faces substantial challenges, including accuracy concerns, high computational costs, integration friction with legacy systems, and ongoing user trust deficits in AI-generated answers.

At the heart of modern AI search is the embedding — a numerical vector representation of text, images, or other content that captures its semantic meaning. Embeddings allow AI systems to find conceptually similar content even when exact keywords do not match, which is precisely the limitation of older lexical search methods. Lexical search matches exact words and phrases; semantic search understands meaning and concepts, retrieving relevant results even when no keywords overlap.

Dense retrieval generalizes this idea by encoding both queries and documents into vector embeddings and matching them via cosine similarity, in contrast to sparse retrieval (such as BM25) which matches exact lexical tokens. BM25 itself remains a strong baseline and is still relevant in 2025 AI pipelines because it excels at exact-match recall on rare terms — a property that dense retrieval alone can struggle to preserve. Dense Passage Retrieval (DPR), introduced by Karpukhin et al. in 2020, was the first dense retriever to outperform BM25 on open-domain question answering, using two BERT encoders (one for the question, one for the passage) trained with contrastive loss on QA pairs.

Modern embedding systems rely on standardized benchmarks to measure progress. The Massive Text Embedding Benchmark (MTEB) covers 56 datasets across retrieval, classification, clustering, and semantic textual similarity, and is the de facto leaderboard for general-purpose embeddings. State-of-the-art open models in 2025 include BGE-M3, E5-Mistral-7B, and Nomic-Embed-Text-v1.5, all of which score above 65 on MTEB for English and offer strong multilingual performance. Typical embedding dimensions today are 384, 768, 1024, or 1536; OpenAI's text-embedding-3-small supports 512 or 1536, while text-embedding-3-large supports up to 3072. Matryoshka Representation Learning (MRL) trains embeddings so that the first \( d \) dimensions form a useful embedding on their own, allowing a single model to serve multiple storage and recall budgets (e.g. 64, 128, 256, 512) without retraining.

The advantages of AI search over keyword search are direct consequences of these technical foundations: AI search understands context, intent, and semantic meaning rather than just matching keywords, providing more relevant and personalized results. This semantic depth is also why Information Retrieval is foundational to broader AI development, particularly the five new retrieval tasks introduced in the context of Artificial General Intelligence — chief among them External Information Retrieval, which allows AI agents to access new information that was not seen during training, enabling continuous learning and up-to-date knowledge retrieval.

Because pure dense retrieval can miss exact lexical matches and pure sparse retrieval can miss semantic intent, the dominant production pattern in 2025 is hybrid search — a combination of BM25 and dense vector retrieval, often with an additional cross-encoder reranking stage. The two ranked lists are typically merged using Reciprocal Rank Fusion (RRF), which sums \( 1 / (k + r_i) \) across retrievers with \( k \) typically set to 60. RRF requires no score calibration and is robust across heterogeneous rankers, which is why it has become the standard merger.

The choice of retriever architecture involves a key trade-off. A bi-encoder encodes queries and documents independently, enabling fast approximate nearest neighbor (ANN) search but with less accuracy. A cross-encoder jointly attends over the query and document, producing much more accurate relevance scores but at a computational cost too high for first-stage retrieval. In practice, bi-encoders handle the initial candidate generation while cross-encoders are reserved for reranking. ColBERT bridges this gap with late interaction: it encodes every token of every document offline and uses MaxSim over query tokens at query time, balancing bi-encoder speed with cross-encoder quality. ColBERTv2 adds residual compression and denormalized embeddings, cutting index size by roughly 10x while improving retrieval quality over the original ColBERT.

Learned sparse retrievers offer another path that is hybrid by design. SPLADE uses BERT's masked language modeling head to predict which vocabulary terms should be activated per document, producing sparse but expanded representations that combine neural semantics with lexical sparsity. Elastic's ELSER (Elastic Learned Sparse EncodeR) is an out-of-domain learned sparse retriever that ships as a managed model in Elasticsearch, giving BM25-style sparse vectors with semantic understanding out of the box.

Reranking applies a cross-encoder — such as bge-reranker-v2-m3 or Cohere Rerank 3.5 — to the top-k candidates from first-stage retrieval, reordering them by fine-grained relevance and typically improving top-1 accuracy by 10-30%. Cohere's API signature, co.rerank(query=..., documents=[...], top_n=k, model="rerank-english-v3.0"), returns indices and relevance scores used to reorder first-stage hits before the LLM step. Reranker distillation trains a small bi-encoder student to mimic the ranking signal of a large cross-encoder teacher, recovering most of the cross-encoder's quality at bi-encoder speed. Finally, Maximal Marginal Relevance (MMR) reorders retrieved documents to maximize relevance while minimizing redundancy, scoring each candidate as \( \lambda \cdot \text{sim}(q, d) - (1 - \lambda) \cdot \max_{d'} \text{sim}(d, d') \) where \( d' \) is already selected. This is especially useful when feeding top-k context into an LLM, where redundant chunks waste tokens and dilute attention.

Retrieval-augmented generation (RAG) combines retrieval of relevant documents with generative AI to produce accurate, context-aware answers grounded in retrieved information. The typical 2025 RAG pipeline flows as: query rewrite → hybrid retrieve (BM25 + dense) → rerank → top-k context → LLM with citations → answer, with optional HyDE, query expansion, and post-generation grounding checks. Several techniques refine the query side: query rewriting uses an LLM to transform a conversational or ambiguous query into a self-contained search query by resolving pronouns and expanding acronyms, dramatically improving recall on multi-turn dialogues; HyDE (Hypothetical Document Embeddings) prompts an LLM to generate a hypothetical answer first, embeds that, and retrieves documents similar to the hypothetical — useful for zero-shot domains where the original query is short; and step-back prompting generates a more abstract or general query alongside the original, retrieves context for both, and feeds both into the LLM, improving answers on factual, multi-hop, or scientific questions.

Multi-hop retrieval is required when a question demands chaining evidence across documents. Solutions include iterative retrieval (retrieve → answer → re-query), IRCoT (Interleaved Retrieval with Chain-of-Thought), and graph-based traversal over linked passages. Microsoft Research introduced GraphRAG in mid-2024, which builds a knowledge graph from documents (entities plus relations) and retrieves subgraphs relevant to the query, often using community detection to answer global questions like "what are the main themes?" — an approach that dramatically outperforms chunk-based RAG on such global sensemaking tasks. The next generation of RAG is agentic: agentic RAG gives the LLM tools and a loop to decide when and what to retrieve, when to re-query, when to call APIs, and when to answer — replacing the rigid retrieve-then-generate pipeline. Self-RAG trains the model to emit special [Retrieve] and [IsRel/IsSup/IsUse] reflection tokens so the LLM dynamically decides whether to retrieve and whether its own outputs are grounded and useful. CRAG (Corrective RAG) adds a lightweight retrieval evaluator that grades retrieved documents as Correct, Ambiguous, or Incorrect and triggers web search or fallback for low-confidence cases.

Four classic RAG failure modes are routinely diagnosed: missing content (not retrieved), missed top-ranked docs, not in context (lost in the middle), and not extracted (LLM ignores the answer even when present). The lost-in-the-middle problem refers to LLMs giving more weight to context at the beginning and end of the prompt and ignoring the middle, which is why rerankers should place the most relevant chunks at the top and bottom of the context window. A common default chunk size for long documents is 512 tokens with 10-20% overlap (~50-100 tokens); character/token chunking is fast but arbitrary, while semantic chunking splits at topic shifts using embedding similarity, producing more coherent chunks at the cost of latency. Two further research directions push beyond plain RAG: retrieval-augmented fine-tuning (RAFT) fine-tunes the LLM on QA pairs that include retrieved and sometimes irrelevant context, teaching it to ignore distractors, while RA-DIT (Retrieval-Augmented Dual Instruction Tuning, Meta 2023) separately fine-tunes the LLM and retriever on instruction data, improving both modules jointly.

The agentic pattern of fan-out — splitting a question into sub-questions, retrieving in parallel, and aggregating results — underlies Deep Research products and multi-hop agentic RAG. The historical roots of agentic retrieval reach back to ReAct (Reasoning + Acting), which prompts the LLM to interleave Thought → Action → Observation steps with actions like Search[query] and Lookup[link], and to Toolformer (Meta 2023), which self-supervises API calls by training only when the call actually improves perplexity. WebGPT (OpenAI 2021) showed that a GPT-3 model fine-tuned to browse the web and cite sources could produce answers preferred over human-written Reddit responses, foreshadowing today's AI search products.

AI search increasingly operates across modalities and languages, not just text. Multimodal AI search can understand and retrieve information across text, images, audio, and video formats. CLIP (Contrastive Language-Image Pretraining) trained a text and image encoder in a shared space using 400 million image-caption pairs, enabling zero-shot image search and text-to-image retrieval. SigLIP replaces CLIP's softmax contrastive loss with a sigmoid loss, scaling better and achieving higher zero-shot ImageNet accuracy at smaller batch sizes — it is now used in Gemini vision encoders. ColPali embeds document page images directly with a vision-language model (PaliGemma) and uses ColBERT-style late interaction, beating OCR-then-text pipelines on visually rich PDFs. Vision-native retrievers like ColPali are particularly valuable because OCR errors propagate to embeddings and BM25 indexes, hurting recall on tables, equations, handwriting, and low-resolution scans.

Beyond multimodality, AI search handles a wide range of specialized challenges. Cross-lingual information retrieval lets AI systems search and retrieve information across multiple languages, understanding queries in one language and finding relevant content in others. Zero-shot and few-shot learning allow search systems to handle new types of queries without extensive retraining, adapting quickly to new domains and languages. When faced with ambiguous queries, AI search engines use context clues, clarification questions, and multiple intent detection to provide relevant results or ask for clarification when needed. Real-time information is accessed through External Information Retrieval, which pulls current data from the web, APIs, and live sources beyond training data. AI search engines improve personalization by learning from user behavior, context, and preferences, and they rely on conversation history, user preferences, and search context to deliver more relevant results over time. Advanced systems provide direct citations and links to original sources, allowing users to verify information and explore further, and combat misinformation through source verification, citation linking, confidence scoring, and retrieval from authoritative sources.

A growing ecosystem of benchmarks tracks these capabilities. FreshQA, a Google benchmark of time-sensitive questions whose answers change, found that modern Gemini-based AI search systems reached roughly 95% accuracy by early 2025. OpenAI's SimpleQA is a short-form factuality benchmark with single-answer questions designed to measure hallucination rates. HALo (Hallucinations and Logical operators) from Vectara tracks hallucination rate vs. summary length; GPT-4 class models run at about 3% hallucination, while earlier models exceeded 20%. For agentic systems, GAIA (Meta 2023) is a benchmark of assistant-style multi-step questions requiring web search, file reading, and calculation — frontier agents now exceed 65% while 2023 GPT-4 sat at roughly 15%. WebArena (2023) is a realistic, self-hostable web agent benchmark built on clones of GitHub, Reddit, Maps, and shopping sites; top agents still complete fewer than 30% of tasks in 2025.

These capabilities are increasingly delivered through deep research agents. Deep Research is a multi-step research agent from OpenAI, Google, and Perplexity that browses 50-200+ sources, runs code, and produces a long-form report with citations in 5-30 minutes. Typical Deep Research tasks cost $0.50 to $5 in API (or 1-25 Pro credits) due to the 100+ web fetches and long reasoning traces — roughly 10-100x a simple RAG query. Perplexity's Sonar model exposes an agentic search loop via API that iteratively queries the web, reads pages, refines the query, and synthesizes a cited answer. However, these agents face real security risks: indirect prompt injection via web content occurs when a website includes invisible text instructing the AI to ignore its priors, exfiltrate user data, or change behavior — a top OWASP LLM risk in 2025. Defenses include treating retrieved content as untrusted data rather than instructions, using delimiters and quoting, performing content-type checks, and routing retrieved text through a separate policy LLM before it reaches the main model.

Running AI search at scale requires sophisticated infrastructure for vector storage, indexing, and serving. Approximate nearest neighbor (ANN) search trades a small loss in recall for huge speed gains over exact kNN by using index structures like HNSW, IVF-PQ, or ScaNN, enabling sub-linear vector lookup over millions of embeddings. Hierarchical Navigable Small World (HNSW) is a graph-based ANN index that builds multi-layer proximity graphs, offering logarithmic-time search, high recall, and incremental inserts — making it the most popular vector index. Product quantization (PQ) splits each embedding into sub-vectors and replaces each with the nearest centroid id from a codebook, compressing vectors by roughly 32x with small recall loss for billion-scale retrieval. DiskANN is a Microsoft graph-based ANN index designed for SSD-resident vectors at billion scale with the Vamana graph, used in Azure AI Search and Bing's vector backend. Meta's FAISS (Facebook AI Similarity Search) is the open-source library for efficient similarity search and clustering of dense vectors, with GPU support and many index types — the toolkit beneath most vector databases. In a one-line comparison: HNSW is graph-based with high recall but more RAM, IVF-PQ is inverted-file plus product quantization with low memory and lower recall, and ScaNN is Google's anisotropic vector quantization that performs at top-tier on TPU/CPU benchmarks.

The vector database landscape offers many choices. Pinecone is a managed serverless vector database (built on proprietary infrastructure) popular for production RAG, offering metadata filtering, namespaces, and hybrid sparse+dense search. Weaviate is an open-source vector DB with built-in vectorization modules (OpenAI, Cohere, HuggingFace) and GraphQL/REST APIs, supporting hybrid search and generative RAG out of the box. Qdrant is a Rust-based open-source vector DB with high-performance HNSW, rich payload filtering, and quantization, widely used for on-prem and self-hosted RAG. Milvus (built by Zilliz) is an open-source, distributed vector DB with multiple index types targeting billion-scale deployments — the OSS counterpart to Pinecone at scale. pgvector is a PostgreSQL extension that adds vector types and ANN search to Postgres, popular for teams that want a single OLTP+vector store without a new system. LanceDB is an embedded, serverless vector DB built on the Lance columnar format, optimized for ML workloads with zero ops. For broader hybrid systems, Elasticsearch combines BM25 plus ELSER plus optional dense_vector in one engine, OpenSearch's neural_search plugin adds dense and sparse neural retrievers, and Vespa.ai is Yahoo's open-source engine for large-scale hybrid search plus ranking plus ML inference. The major difference between vector DBs and traditional search engines is that traditional engines optimize lexical recall and aggregations while vector DBs optimize semantic similarity at scale via ANN; hybrid systems increasingly offer both but trade-offs in freshness, filter expressiveness, and cost remain.

Distance metric selection matters: cosine is standard for normalized embeddings, dot product for unnormalized ones, and Euclidean for absolute-distance clustering. Metadata filters (e.g. tenant_id, date > 2024, language = "en") are pre-filters that restrict ANN search to a subset of vectors, critical for multi-tenant and time-bounded retrieval. The SIFT-1B benchmark (1 billion 128-dim SIFT descriptors) is the standard large-scale ANN benchmark; state-of-the-art systems achieve greater than 95% recall@10 at sub-millisecond latency per query on a single GPU. Latency budgets are tight in production: a typical RAG query targets about 1-3 seconds end-to-end (roughly 50-150ms retrieval, 100-300ms reranking, and 500-2000ms for the LLM to stream its first token), and anything over 3 seconds drops user engagement sharply. With HNSW on modern hardware, p95 hybrid search over 10M vectors runs in roughly 50-150ms, well within the RAG latency budget.

Optimizing cost and context length is just as important as optimizing latency. Embedding too many chunks per query wastes tokens and dilutes attention; it is better to retrieve, then compress or re-rank down to the 5-15 most relevant chunks before the LLM. LongLLMLingua (LLMLingua) uses a small model to remove non-essential tokens from retrieved documents (adjectives, restatements), achieving 4-20x compression with minimal quality loss. Anthropic's 2024 "contextual retrieval" technique prepends a 50-token LLM-generated context summary to each chunk before embedding, reducing retrieval failure rate by 35-49%. Scalar (int8) or binary (1-bit) quantization of embeddings cuts vector DB storage and compute by 4-32x, with typical recall drop of 1-3%, often offset by adding a reranker. Prompt caching, now offered by OpenAI, Anthropic, and Google, reuses a cached prefix for 5-10 minutes, cutting cost up to 90% and latency up to 80%; semantic caching stores previous (query, embedding, answer) tuples and returns a cached answer if a new query is within a similarity threshold of an old one. Token streaming returns LLM tokens as they are generated via server-sent events, so the user sees the answer build up in real time even when the full answer takes 5-10 seconds to complete. The UI also matters: AI search uses an "answer-first" design with the generated answer at the top followed by citations, sources, related questions, and traditional links — the inverse of classic "10 blue links" SERPs. Users now read the AI answer first and click through to verify 1-2 sources; the "zero-click" share of search queries is above 65% per SparkToro (2024).

The consumer AI search product landscape is dominated by a small number of well-funded players. Perplexity is an answer engine that pairs a generative LLM with live web search and inline numbered citations on every sentence, positioning itself as a "Google plus ChatGPT" hybrid; Perplexity Pages (2024) lets users turn a search session into a shareable, citable article, blurring the line between search and publishing. OpenAI integrated real-time web search into ChatGPT in late 2024 (SearchGPT), returning conversational answers with source links for queries with recency intent. Google's AI Overviews (formerly SGE) are generative summaries that appear above traditional blue-link results, launched broadly in May 2024 and powered by a customized Gemini model. Google's 2025 "AI Mode" goes further: a full conversational, Gemini-powered search interface that returns only AI answers with follow-up, replacing the ten blue links. Microsoft Copilot (formerly Bing Chat) combines GPT-4-class models with Bing's index to deliver cited, conversational answers, integrated across Edge, Windows, and Microsoft 365. You.com offers a multi-modal, multi-agent answer engine with customizable apps (YouAgents) and modes (Smart, Genius, Research). Brave Search's Leo is a private, no-log AI answer assistant inside the browser; Brave's premium Leo subscription (~$15/mo) and its ad-free index differentiate it from Google-dependent competitors. ChatGPT receives an estimated 5-6% of global search referrals and is the most-used AI search destination per Similarweb and SparkToro analyses in 2025. Google's "Circle to Search" feature, announced on Pixel 8 and Samsung Galaxy S24 in 2024, lets users circle any on-screen object to trigger a visual AI search combining image and text context.

This new generation of products requires a new optimization discipline. GEO (Generative Engine Optimization) optimizes content to be cited inside LLM-generated answers — clear structure, statistics, authoritative sources — rather than just ranked in blue links, with new metrics like citation share and answer inclusion rate. The Princeton and IIT Delhi GEO paper (2023/2024) coined the term and showed that techniques like citing sources, adding statistics, and using quotations can boost source visibility in LLM answers by up to 40%. Schema.org markup is a vocabulary of structured data (FAQ, Product, Article) embedded in HTML; AI search engines parse it for entity disambiguation, prices, dates, and citation-ready facts, boosting GEO inclusion. The proposed llms.txt standard is a markdown index of a site's high-quality LLM-readable content, placed at /llms.txt, so AI crawlers can ingest a concise site summary instead of crawling thousands of HTML pages. Site owners can block GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, CCBot, and PerplexityBot via standard robots.txt; a User-agent: GPTBot / Disallow: / rule blocks both training and retrieval. There is an important distinction between training crawls (e.g. GPTBot, CCBot), which ingest content for model training, and retrieval crawls (e.g. OAI-SearchBot, ChatGPT-User), which fetch pages live to ground answers and obey per-page fetch policies.

The legal landscape is evolving rapidly. OpenAI, Microsoft, and Google have signed content licensing agreements with AP, Axel Springer, News Corp, the Financial Times, The Atlantic, Vox, Le Monde, and Reddit (~$60M/yr) for both training data and live retrieval rights. The New York Times sued OpenAI and Microsoft in December 2023 alleging that ChatGPT and Copilot reproduce its articles verbatim and that OpenAI used its content to build a substitute product, seeking statutory and actual damages. Under the EU DSM Directive Article 4, rights holders can reserve use of their works for text and data mining (TDM) via machine-readable means, which governs how EU-based AI search engines must respect commercial opt-outs. The impact on publishers is severe: multiple analyses (Authoritas, 2024) estimate AI Overviews and ChatGPT search could cut publisher organic traffic 20-60% for informational queries, prompting affiliate- and direct-traffic pivots. Counter-strategies include content licensing, defending via robots.txt and paywalls, and building direct AI products (Yahoo News AI, Hearst Aria).

Enterprise AI search adds further layers of complexity. Unlike consumer AI search, which optimizes for freshness, coverage, and cost, enterprise AI search requires access control (ACLs), audit logs, connectors to SharePoint, Confluence, and Slack, hybrid indexes, and provenance. Glean is an AI search and assistant for the enterprise that indexes Slack, Google Drive, Confluence, GitHub, Salesforce, and more, respects per-document ACLs, and grounds answers in source links. A typical 2025 enterprise RAG architecture flows: connectors (M365, Slack, Drive) → chunker → embedding plus BM25 index → ACL-aware retriever → reranker → grounded LLM with citations → audit log and provenance, with separate evaluation and feedback loops. Permissions-aware retrieval filters results to documents the current user can access; solutions include passing ACL as metadata filters, indexing with user/group claims, and post-filtering on access tokens. Answer provenance tracks the exact document, version, and span that contributed each claim, so audits can reproduce the answer — required in finance, pharma, and legal AI deployments. Many enterprise answers live in SQL, knowledge graphs, or APIs rather than documents, so modern AI search combines text-to-SQL, KG-RAG, and tool-calling agents to retrieve structured facts alongside unstructured passages. A knowledge graph (KG) stores entities and relations (e.g. (Tesla, founded_by, Elon_Musk)); retrieval over a KG supports symbolic, multi-hop reasoning that pure vector search cannot. Wikidata underpins many AI search answers with canonical entity IDs and relations, used directly by Google Knowledge Graph, Apple's Siri, and RAG systems needing ground-truth facts. The connective tissue across all of these systems is the Model Context Protocol (MCP), introduced by Anthropic in 2024 as a standardized JSON-RPC protocol for connecting LLMs to tools and data sources (search, files, DBs), enabling plug-and-play AI search integrations across clients and vendors. Function calling (or tool use) is the underlying mechanism by which models emit structured JSON tool calls (e.g. {"tool": "search", "args": {"query": "..."}}) that the runtime executes and returns to the model. Query expansion adds synonyms and paraphrases to broaden recall, while query decomposition breaks one question into multiple sub-questions, each retrieved independently and then combined. Small language models (SLMs) like Phi-4, Gemma 2 9B, and Llama 3.1 8B increasingly handle query rewriting, routing, and reranking at the edge, leaving only the final generation to a frontier model — cutting cost 10-50x. Compared to long-context LLMs that fit everything in the prompt but are expensive, slow, and still miss mid-document facts, RAG retrieves only relevant spans and is cheaper, faster, and more auditable per query — making it the default architecture for AI search in 2025 and beyond.