How does hybrid search improve RAG?
- 1 min — why recall at candidate generation is the unrecoverable stage.
- 2 min — fusion: RRF default, weighted fusion when labelled data exists.
- 2 min — cost: second index, sync, single-engine option.
- 1 min — evidence: per-leg vs fused Recall@50, candidate overlap.
It runs both retrievers and fuses the results, so a document found by either method enters the candidate pool. The gain is recall at the candidate-generation stage, which is the stage you can't recover from — a reranker can only reorder what retrieval handed it.
Fusion method matters. Reciprocal Rank Fusion combines by rank position, needs no score normalisation, and is robust — make it your default. Weighted score fusion can be better tuned but requires normalising two incomparable score scales and re-tuning whenever either retriever changes.
Hybrid costs you a second index to operate, keep in sync, and pay for. Say that. The honest framing: "Hybrid plus a reranker is the highest-value standard architecture, and in my experience the hybrid part contributes more than the reranker when queries contain identifiers, and less when they're conversational."
Org scope: prefer an engine that does both legs natively (OpenSearch, Elasticsearch, Qdrant, Weaviate, or Postgres with pgvector plus full-text) over two separate stores with a sync job — one ingestion path, one owner, one on-call rotation.
BM25
- Doc A
- Doc C
- Doc F
Dense
- Doc B
- Doc A
- Doc D
Fused (RRF)
- Doc A
- Doc B
- Doc C
- Doc D
What the answer sounds like at each level
Run keyword and vector search, merge the lists with RRF, and send the merged candidates to the LLM or reranker.
Explains RRF (score = Σ 1/(k + rank), k = 60), why it avoids score normalisation, and measures Recall@50 for each leg alone vs fused to show what hybrid buys.
Weighs the second index's operating cost against the measured recall gain per query segment, chooses a single engine that serves both legs, and owns sync consistency between them.
Follow-ups the interviewer will push
1Fused recall is barely above dense alone. What do you conclude?
The lexical leg isn't contributing unique hits for this traffic — check the overlap of the two candidate sets. If >85% overlap and identifier queries are rare, hybrid may not earn its cost; keep BM25 only for the segment that needs it, routed by a cheap classifier.
2When would you choose weighted score fusion over RRF?
When you have enough labelled queries (a few hundred+) to tune the weight and both retrievers are stable. Normalise each leg (min-max per query or z-score), sweep α on the golden set, and re-tune whenever either model changes. Otherwise RRF with k = 60.
3The two indexes drift: a document is in BM25 but not the vector index. How do you prevent that?
Single ingestion pipeline writing both from the same registry record, idempotent upserts keyed by chunk id and content hash, and a nightly reconciliation job that diffs ids across the two and alerts on mismatch above 0.1%.
Red flags interviewers listen for
- Adds raw BM25 and cosine scores together without normalisation.
- Ignores the operational cost and consistency risk of a second index.
- Can't say what RRF's k parameter does.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Recall reasoning | Locates hybrid's value at candidate generation and explains why a reranker can't fix missing candidates. |
| Fusion | Describes RRF precisely and knows when weighted fusion is worth the tuning burden. |
| Cost & ops | Names the second index's cost and a concrete consistency control. |
Sources
- Reciprocal Rank Fusion outperforms Condorcet and individual Rank Learning Methods — Cormack et al., 2009paper
- The Probabilistic Relevance Framework: BM25 and Beyond — Robertson & Zaragoza, 2009paper