Skip to content

Chapter 2 of 7

Neural Retrieval: Embeddings, Vectors & Semantic Search

At the heart of modern AI search is the embedding — a numerical vector representation of text, images, or other content that captures its semantic meaning. Embeddings allow AI systems to find conceptually similar content even when exact keywords do not match, which is precisely the limitation of older lexical search methods. Lexical search matches exact words and phrases; semantic search understands meaning and concepts, retrieving relevant results even when no keywords overlap.

Dense retrieval generalizes this idea by encoding both queries and documents into vector embeddings and matching them via cosine similarity, in contrast to sparse retrieval (such as BM25) which matches exact lexical tokens. BM25 itself remains a strong baseline and is still relevant in 2025 AI pipelines because it excels at exact-match recall on rare terms — a property that dense retrieval alone can struggle to preserve. Dense Passage Retrieval (DPR), introduced by Karpukhin et al. in 2020, was the first dense retriever to outperform BM25 on open-domain question answering, using two BERT encoders (one for the question, one for the passage) trained with contrastive loss on QA pairs.

Modern embedding systems rely on standardized benchmarks to measure progress. The Massive Text Embedding Benchmark (MTEB) covers 56 datasets across retrieval, classification, clustering, and semantic textual similarity, and is the de facto leaderboard for general-purpose embeddings. State-of-the-art open models in 2025 include BGE-M3, E5-Mistral-7B, and Nomic-Embed-Text-v1.5, all of which score above 65 on MTEB for English and offer strong multilingual performance. Typical embedding dimensions today are 384, 768, 1024, or 1536; OpenAI's text-embedding-3-small supports 512 or 1536, while text-embedding-3-large supports up to 3072. Matryoshka Representation Learning (MRL) trains embeddings so that the first \( d \) dimensions form a useful embedding on their own, allowing a single model to serve multiple storage and recall budgets (e.g. 64, 128, 256, 512) without retraining.

The advantages of AI search over keyword search are direct consequences of these technical foundations: AI search understands context, intent, and semantic meaning rather than just matching keywords, providing more relevant and personalized results. This semantic depth is also why Information Retrieval is foundational to broader AI development, particularly the five new retrieval tasks introduced in the context of Artificial General Intelligence — chief among them External Information Retrieval, which allows AI agents to access new information that was not seen during training, enabling continuous learning and up-to-date knowledge retrieval.

All chapters
  1. 1The AI Search Revolution: Market & Fundamentals
  2. 2Neural Retrieval: Embeddings, Vectors & Semantic Search
  3. 3Hybrid Search, Reranking & Sparse-Dense Bridges
  4. 4RAG Architectures: From Pipelines to Agentic Systems
  5. 5Multimodal, Specialized & Benchmarked Capabilities
  6. 6Production Infrastructure: Vector Databases, Latency & Cost
  7. 7The Ecosystem: Products, GEO, Legal & Enterprise

Drill it

Reading is not remembering. These come from the AI Search Results Anki deck:

Q

What percentage of the combined search market does AI search account for as of June 2025?

AI search traffic accounts for 7.82% of the combined search market as of June 2025.

Q

What is Neural Information Retrieval?

Neural Information Retrieval is an AI approach where search engines use neural networks to understand and retrieve information more efficiently, forming a corne...

Q

What are the five new retrieval tasks introduced for Artificial General Intelligence?

The five new retrieval tasks include External Information Retrieval, which allows AI agents to access new information that was not seen during training.

Q

What is the forecast period for when AI search might surpass Google?

The forecast extends from 2025 to 2030, projecting when ChatGPT and AI-powered search will surpass traditional search engines like Google.