Because pure dense retrieval can miss exact lexical matches and pure sparse retrieval can miss semantic intent, the dominant production pattern in 2025 is hybrid search — a combination of BM25 and dense vector retrieval, often with an additional cross-encoder reranking stage. The two ranked lists are typically merged using Reciprocal Rank Fusion (RRF), which sums \( 1 / (k + r_i) \) across retrievers with \( k \) typically set to 60. RRF requires no score calibration and is robust across heterogeneous rankers, which is why it has become the standard merger.
The choice of retriever architecture involves a key trade-off. A bi-encoder encodes queries and documents independently, enabling fast approximate nearest neighbor (ANN) search but with less accuracy. A cross-encoder jointly attends over the query and document, producing much more accurate relevance scores but at a computational cost too high for first-stage retrieval. In practice, bi-encoders handle the initial candidate generation while cross-encoders are reserved for reranking. ColBERT bridges this gap with late interaction: it encodes every token of every document offline and uses MaxSim over query tokens at query time, balancing bi-encoder speed with cross-encoder quality. ColBERTv2 adds residual compression and denormalized embeddings, cutting index size by roughly 10x while improving retrieval quality over the original ColBERT.
Learned sparse retrievers offer another path that is hybrid by design. SPLADE uses BERT's masked language modeling head to predict which vocabulary terms should be activated per document, producing sparse but expanded representations that combine neural semantics with lexical sparsity. Elastic's ELSER (Elastic Learned Sparse EncodeR) is an out-of-domain learned sparse retriever that ships as a managed model in Elasticsearch, giving BM25-style sparse vectors with semantic understanding out of the box.
Reranking applies a cross-encoder — such as bge-reranker-v2-m3 or Cohere Rerank 3.5 — to the top-k candidates from first-stage retrieval, reordering them by fine-grained relevance and typically improving top-1 accuracy by 10-30%. Cohere's API signature, co.rerank(query=..., documents=[...], top_n=k, model="rerank-english-v3.0"), returns indices and relevance scores used to reorder first-stage hits before the LLM step. Reranker distillation trains a small bi-encoder student to mimic the ranking signal of a large cross-encoder teacher, recovering most of the cross-encoder's quality at bi-encoder speed. Finally, Maximal Marginal Relevance (MMR) reorders retrieved documents to maximize relevance while minimizing redundancy, scoring each candidate as \( \lambda \cdot \text{sim}(q, d) - (1 - \lambda) \cdot \max_{d'} \text{sim}(d, d') \) where \( d' \) is already selected. This is especially useful when feeding top-k context into an LLM, where redundant chunks waste tokens and dilute attention.