When would you use dense retrieval vs sparse retrieval?
- 1 min — define both by mechanism (term match vs vector similarity).
- 2 min — failure case of each with a concrete query.
- 2 min — decision rule by corpus and user type; default to hybrid.
- 1 min — how you'd prove it: per-segment Recall@50 vs BM25 baseline.
Sparse (BM25, SPLADE) matches terms. It's exact, interpretable, needs no training, and handles rare tokens — error codes, SKUs, person names, acronyms — perfectly. It fails on vocabulary mismatch. Dense matches meaning. It handles paraphrase and synonymy, and fails on exact identifiers it never saw in training: a query for ERR_4821 retrieves things that look vaguely like error codes.
Reach for sparse-only when your corpus is jargon-dense and your users are experts who already use the right terms — legal search, code search. Reach for dense-only when queries are conversational and your corpus is small enough that lexical recall doesn't matter. Note that BM25 is extremely cheap and a startlingly strong baseline; any dense system should be required to beat it on your own eval before it ships.
Org scope: the retrieval platform team owns the BM25 baseline and the shared eval set; product teams own their query logs. Buy the lexical engine (OpenSearch/Elasticsearch or Postgres full-text) rather than building it, and make "beats BM25 on our golden set" a gate every team's dense rollout has to pass.
Sparse (BM25)
- Exact term match
- IDs, SKUs, codes
- No training needed
- Misses synonyms
Dense (embeddings)
- Meaning match
- Paraphrase, intent
- Model + GPU to encode
- Misses rare tokens
What the answer sounds like at each level
Sparse matches exact words, dense matches meaning; use BM25 for identifiers and dense for paraphrased questions, and usually both.
Quantifies it on the team's own data: run BM25, a dense model (e.g. bge-m3 or text-embedding-3-small) and hybrid on a 200–500 query golden set, report Recall@50 per query type (identifier vs natural language), and picks per segment.
Treats BM25 as the permanent baseline and regression canary, decides which query segments justify the cost of a second index, and owns the eval gate that every team's dense model must clear before shipping.
Follow-ups the interviewer will push
1Your corpus is 40% source code and stack traces. What changes?
Lexical becomes the primary leg: identifiers, function names and error strings dominate. Keep BM25 with a code-aware tokenizer (split camelCase and snake_case), add a code-trained embedding as the second leg, and fuse with RRF. Measure separately on identifier queries, where dense alone often loses 20–40 points of recall.
2Now the queries are in five languages. Does BM25 still hold up?
Only per language: BM25 needs per-language analyzers (stemming, CJK segmentation) and can't match a German query to an English document. A multilingual dense model (bge-m3, multilingual-e5) handles cross-lingual matching, so dense's share of the fused list should rise. Evaluate per language, not in aggregate.
3Where does SPLADE fit?
It's a learned sparse model: it expands terms (adds 'jacket' weight to 'parka') while staying on an inverted index. Use it when you want dense-like recall on lexical infrastructure, and accept higher index size (2–4× BM25) and a GPU for encoding queries.
Red flags interviewers listen for
- Claims dense retrieval strictly dominates BM25 without measuring on their own corpus.
- Never mentions identifiers, SKUs or error codes as the dense failure case.
- Recommends one approach without naming what would change their mind.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Mechanism | Explains term matching vs embedding similarity and why each fails where it does. |
| Decision | Gives a conditional rule (expert jargon → sparse, conversational → dense, mixed → hybrid). |
| Measurement | Insists on a BM25 baseline and per-query-type recall on the team's own golden set. |
Sources
- The Probabilistic Relevance Framework: BM25 and Beyond — Robertson & Zaragoza, 2009paper
- Dense Passage Retrieval for Open-Domain Question Answering — Karpukhin et al., 2020paper
- SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking — Formal et al., 2021paper