The consumer AI search product landscape is dominated by a small number of well-funded players. Perplexity is an answer engine that pairs a generative LLM with live web search and inline numbered citations on every sentence, positioning itself as a "Google plus ChatGPT" hybrid; Perplexity Pages (2024) lets users turn a search session into a shareable, citable article, blurring the line between search and publishing. OpenAI integrated real-time web search into ChatGPT in late 2024 (SearchGPT), returning conversational answers with source links for queries with recency intent. Google's AI Overviews (formerly SGE) are generative summaries that appear above traditional blue-link results, launched broadly in May 2024 and powered by a customized Gemini model. Google's 2025 "AI Mode" goes further: a full conversational, Gemini-powered search interface that returns only AI answers with follow-up, replacing the ten blue links. Microsoft Copilot (formerly Bing Chat) combines GPT-4-class models with Bing's index to deliver cited, conversational answers, integrated across Edge, Windows, and Microsoft 365. You.com offers a multi-modal, multi-agent answer engine with customizable apps (YouAgents) and modes (Smart, Genius, Research). Brave Search's Leo is a private, no-log AI answer assistant inside the browser; Brave's premium Leo subscription (~$15/mo) and its ad-free index differentiate it from Google-dependent competitors. ChatGPT receives an estimated 5-6% of global search referrals and is the most-used AI search destination per Similarweb and SparkToro analyses in 2025. Google's "Circle to Search" feature, announced on Pixel 8 and Samsung Galaxy S24 in 2024, lets users circle any on-screen object to trigger a visual AI search combining image and text context.
This new generation of products requires a new optimization discipline. GEO (Generative Engine Optimization) optimizes content to be cited inside LLM-generated answers — clear structure, statistics, authoritative sources — rather than just ranked in blue links, with new metrics like citation share and answer inclusion rate. The Princeton and IIT Delhi GEO paper (2023/2024) coined the term and showed that techniques like citing sources, adding statistics, and using quotations can boost source visibility in LLM answers by up to 40%. Schema.org markup is a vocabulary of structured data (FAQ, Product, Article) embedded in HTML; AI search engines parse it for entity disambiguation, prices, dates, and citation-ready facts, boosting GEO inclusion. The proposed llms.txt standard is a markdown index of a site's high-quality LLM-readable content, placed at /llms.txt, so AI crawlers can ingest a concise site summary instead of crawling thousands of HTML pages. Site owners can block GPTBot (OpenAI), ClaudeBot (Anthropic), Google-Extended, CCBot, and PerplexityBot via standard robots.txt; a User-agent: GPTBot / Disallow: / rule blocks both training and retrieval. There is an important distinction between training crawls (e.g. GPTBot, CCBot), which ingest content for model training, and retrieval crawls (e.g. OAI-SearchBot, ChatGPT-User), which fetch pages live to ground answers and obey per-page fetch policies.
The legal landscape is evolving rapidly. OpenAI, Microsoft, and Google have signed content licensing agreements with AP, Axel Springer, News Corp, the Financial Times, The Atlantic, Vox, Le Monde, and Reddit (~$60M/yr) for both training data and live retrieval rights. The New York Times sued OpenAI and Microsoft in December 2023 alleging that ChatGPT and Copilot reproduce its articles verbatim and that OpenAI used its content to build a substitute product, seeking statutory and actual damages. Under the EU DSM Directive Article 4, rights holders can reserve use of their works for text and data mining (TDM) via machine-readable means, which governs how EU-based AI search engines must respect commercial opt-outs. The impact on publishers is severe: multiple analyses (Authoritas, 2024) estimate AI Overviews and ChatGPT search could cut publisher organic traffic 20-60% for informational queries, prompting affiliate- and direct-traffic pivots. Counter-strategies include content licensing, defending via robots.txt and paywalls, and building direct AI products (Yahoo News AI, Hearst Aria).
Enterprise AI search adds further layers of complexity. Unlike consumer AI search, which optimizes for freshness, coverage, and cost, enterprise AI search requires access control (ACLs), audit logs, connectors to SharePoint, Confluence, and Slack, hybrid indexes, and provenance. Glean is an AI search and assistant for the enterprise that indexes Slack, Google Drive, Confluence, GitHub, Salesforce, and more, respects per-document ACLs, and grounds answers in source links. A typical 2025 enterprise RAG architecture flows: connectors (M365, Slack, Drive) → chunker → embedding plus BM25 index → ACL-aware retriever → reranker → grounded LLM with citations → audit log and provenance, with separate evaluation and feedback loops. Permissions-aware retrieval filters results to documents the current user can access; solutions include passing ACL as metadata filters, indexing with user/group claims, and post-filtering on access tokens. Answer provenance tracks the exact document, version, and span that contributed each claim, so audits can reproduce the answer — required in finance, pharma, and legal AI deployments. Many enterprise answers live in SQL, knowledge graphs, or APIs rather than documents, so modern AI search combines text-to-SQL, KG-RAG, and tool-calling agents to retrieve structured facts alongside unstructured passages. A knowledge graph (KG) stores entities and relations (e.g. (Tesla, founded_by, Elon_Musk)); retrieval over a KG supports symbolic, multi-hop reasoning that pure vector search cannot. Wikidata underpins many AI search answers with canonical entity IDs and relations, used directly by Google Knowledge Graph, Apple's Siri, and RAG systems needing ground-truth facts. The connective tissue across all of these systems is the Model Context Protocol (MCP), introduced by Anthropic in 2024 as a standardized JSON-RPC protocol for connecting LLMs to tools and data sources (search, files, DBs), enabling plug-and-play AI search integrations across clients and vendors. Function calling (or tool use) is the underlying mechanism by which models emit structured JSON tool calls (e.g. {"tool": "search", "args": {"query": "..."}}) that the runtime executes and returns to the model. Query expansion adds synonyms and paraphrases to broaden recall, while query decomposition breaks one question into multiple sub-questions, each retrieved independently and then combined. Small language models (SLMs) like Phi-4, Gemma 2 9B, and Llama 3.1 8B increasingly handle query rewriting, routing, and reranking at the edge, leaving only the final generation to a frontier model — cutting cost 10-50x. Compared to long-context LLMs that fit everything in the prompt but are expensive, slow, and still miss mid-document facts, RAG retrieves only relevant spans and is cheaper, faster, and more auditable per query — making it the default architecture for AI search in 2025 and beyond.