Hybrid search
Run keyword (BM25) and vector search in parallel and merge the lists, usually with RRF, so each covers the other's misses.
Hybrid search asks two search engines the same question: a keyword one (BM25) that is good at exact words and codes, and a meaning one (Dense retrieval) that is good at paraphrases. It then merges the two result lists into one. The simplest merge, Reciprocal Rank Fusion (RRF), gives each document points based on where it ranked in each list, so documents that both engines like rise to the top.
Asking both a librarian who knows the catalogue by heart and a well-read friend who knows what books are about, then taking the books both of them recommend first.
Fusing too-short lists. If each leg returns only 10, a document ranked 11th by both engines (strong consensus) is never seen. Retrieve 50–100 per leg, fuse, then cut — the fusion window, not the final k, decides recall.
Both legs retrieve independently (BM25 top 50–100, ANN top 50–100), then fuse. Reciprocal Rank Fusion scores each doc Σ 1/(k + rank) with k=60 (Cormack et al. 2009; Elasticsearch rank_constant default 60, rank_window_size default 100). RRF needs no score calibration, which is why it is the default; weighted score fusion with per-query min-max normalisation can beat it by 1–3 nDCG points once you have labels to tune α. Native support as of Oct 2026: Elasticsearch rrf retriever, OpenSearch hybrid query with a normalization search pipeline, Qdrant Query API fusion: rrf, Vespa rank profiles, Weaviate hybrid with alpha. In Postgres, run a tsvector/BM25 CTE and a pgvector CTE and fuse in SQL. Anthropic's Contextual Retrieval study reported that adding BM25 to contextual embeddings cut top-20 retrieval failures by 49%, and 67% with a reranker on top.
BM25 top 100 + dense top 100, RRF with k=60, cross-encoder (bge-reranker-v2-m3, 568M) on the fused top 50, 5–10 chunks to the LLM. Use the engine's native hybrid (Elasticsearch rrf retriever, OpenSearch hybrid query, Qdrant fusion: rrf) rather than fusing in application code.
Internal wiki, 1.2M chunks. Query 'how do I rotate the PAYMENTS_API_KEY in staging'. BM25 ranks the runbook containing the exact env var at #1 but buries the general 'secret rotation' guide at #40. Dense ranks the guide #2 and the runbook #18. RRF (k=60): runbook = 1/61 + 1/78 = 0.0292; guide = 1/100 + 1/62 = 0.0261; a generic 'staging environments' page ranked #5 by dense only = 1/65 = 0.0154. Both useful docs land in the top 3; a Cross-encoder then orders them.
State a concrete pipeline: "BM25 top 100 + dense top 100, RRF k=60, cross-encoder on the fused top 50, send 8 chunks to the model." Then say how you'd know it helped: recall@50 per leg and fused on a 300-query golden set, broken down by query class (identifier, paraphrase, multi-hop). The follow-up is usually 'why not weighted sum?' — because scores are on different scales; tune α only once you have labels.
BM25
- Rotation runbook
- Staging env vars
- PAYMENTS SDK
- Key mgmt policy
Dense
- Key mgmt policy
- Secret rotation guide
- Rotation runbook
- Staging envs
RRF, k=60
- Rotation runbook
- Key mgmt policy
- Secret rotation guide
- Staging env vars
Numbers worth knowing
- RRF constant
- k = 60Cormack et al. 2009; Elasticsearch default rank_constant
- Retrieval failures, top-20
- −49% with BM25 added; −67% with rerankingAnthropic Contextual Retrieval study, 2024, vs contextual embeddings alone
- Extra latency of the second leg
- +5–20 mslegs run in parallel on the same cluster
How to decide
| When | Do this |
|---|---|
| No relevance labels yet | Use Reciprocal Rank Fusion (RRF) k=60; it needs no tuning. |
| ≥ 300 labelled queries and one leg is clearly stronger | Try weighted Weighted score fusion with per-query min-max; keep only if it beats RRF by ≥ 1 nDCG point. |
| Corpus has no identifiers and queries are pure paraphrase | Dense-only plus reranker is acceptable — but measure the BM25 leg first; it is cheap. |
Cost maths
Hybrid adds the cheaper leg: a BM25 index ~30% of text size and no per-query model cost. For 10M chunks: ~5 GB inverted index vs ~31 GB of 768-d float32 vectors. Latency cost ≈ max(leg) + fusion (< 1 ms), not the sum, if legs run in parallel. The reranker dominates: 50 pairs × ~3 ms on an L4 ≈ 150 ms.
Operating it
- One leg contributes < 5% of final top-10 results for a month — consider dropping or re-weighting it
- Fused recall@50 plateaus while the reranker's top-10 precision is fine — widen rank_window_size to 200
- Hybrid p95 > 150 ms before reranking — legs are running sequentially; parallelise
- Share of final top-10 coming from each leg (or both) — alert if any leg falls below 10%
- Recall@50 fused vs per-leg on the golden set — alert if fused < max(leg)
- End-to-end retrieval p95 — alert above 150 ms excluding rerank
When it goes wrong
Hybrid search on a customer-support knowledge base performed worse than dense alone after launch.
Offline A/B on the golden set: nDCG@10 0.58 hybrid vs 0.63 dense-only.
The team fused with a raw weighted sum (0.5·BM25 + 0.5·cosine); BM25 scores of 8–25 swamped cosine scores of 0.2–0.9, so the result was effectively BM25-only.
Switched to RRF k=60; nDCG@10 rose to 0.69. Added a unit test asserting each leg contributes to the fused top 10 on a fixture query set.
Related terms
Where it shows up
Sources
- Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods — Cormack et al., 2009paper
- Elasticsearch reference: Reciprocal rank fusion — Elastic, 2026docs
- Introducing Contextual Retrieval — Anthropic, 2024blog
Check yourself
Three questions. Every option is explained once you answer.