Sparse retrieval
Keyword search over an inverted index: match the exact words in the query. Fast, explainable, and the best tool for IDs, codes, names and citations.
Sparse retrieval finds documents that contain the same words as your question. Each document is stored as a list of the words it contains, each with a weight; every other word in the language has weight zero, which is why the representation is called sparse. An inverted index maps each word to the documents that contain it, so a lookup touches only the documents that share a word with the query. BM25 is the classic weighting formula and SPLADE is a learned one.
The index at the back of a textbook: look up a word and get the exact pages it appears on. It will never send you to a page that says the same thing in different words.
Treating keyword search as legacy. On BEIR, BM25 averages roughly 0.42–0.44 nDCG@10 zero-shot and the original DPR model scored below it on most out-of-domain datasets. Ripping out the inverted index when adding a vector store is the most common self-inflicted recall regression in RAG systems.
A document is a vector over the vocabulary (30k WordPiece tokens for SPLADE, millions of distinct terms for a Lucene analyzer) where only present terms are non-zero. A query walks one posting list per query term and accumulates per-term scores; dynamic pruning (WAND, Block-Max WAND, on by default in Lucene 8+ when exact hit counts are not requested) skips documents that cannot enter the Top-k (retrieval k vs context k). Cost scales with query length and posting-list length, not corpus dimension, so a single Elasticsearch/OpenSearch node answers a 3-term query over 10M passages in 5–20 ms. The analyzer chain (tokenizer, lowercasing, stemming, synonyms, stop words) decides what counts as a match and moves quality more than any score tweak. The structural weakness is vocabulary mismatch: automobile never matches car unless a synonym list, query rewrite or learned expansion bridges it.
Elasticsearch or OpenSearch with BM25 (k1=1.2, b=0.75 defaults), a standard analyzer on prose fields and a separate code-aware or keyword sub-field for identifiers. In Postgres-only stacks, tsvector + GIN for keywords alongside pgvector, or ParadeDB/pg_search when you need real BM25 scoring.
Code search over a 40M-line monorepo: the query ERR_TLS_CERT_ALTNAME_INVALID hits 7 files through one posting list in 4 ms. A dense retriever on the same query returns 20 generic TLS-config files, because a 768-d embedding spreads that rare token's signal across the whole vector. Keep the identifier field on a keyword or code-aware analyzer (split on _ and camelCase, no stemming) or the exact match degrades too.
If the corpus has identifiers (SKUs, error codes, statute numbers, ticker symbols), say you keep a lexical leg even when you add embeddings, and show you know the analyzer matters: "BM25 with a code-aware analyzer is my exact-match floor; dense handles paraphrase; Hybrid search gets both." Expect the follow-up: how do you combine the scores? Answer Reciprocal Rank Fusion (RRF), because BM25 scores are unbounded and not comparable to cosine.
| Query | Sparse (BM25) | Dense (768-d) |
|---|---|---|
| ERR_TLS_CERT_ALTNAME | hit @1 | miss (rank 30+) |
| 10-K item 1A FY2025 | hit @1 | hit @6 |
| can't log into wifi box | miss | hit @2 |
| contracts with no auto-renew | wrong (ignores 'no') | wrong (ignores 'no') |
Numbers worth knowing
- Query latency, 3-term query
- 5–20 ms p5010M passages, one 8-vCPU Elasticsearch node, warm page cache
- Index size
- 20–40% of raw textLucene postings + positions, no stored source
- BEIR zero-shot nDCG@10
- ≈0.42–0.44 averageBM25 across the BEIR datasets (Thakur et al. 2021)
How to decide
| When | Do this |
|---|---|
| Corpus has identifiers, codes, names, citations | Keep sparse as a mandatory leg of Hybrid search; never dense-only. |
| Queries are conversational paraphrases of prose | Lead with Dense retrieval; keep BM25 as the second leg fused with Reciprocal Rank Fusion (RRF). |
| You want semantic matching but cannot run a vector store | Try SPLADE or Elastic's learned sparse model on the existing inverted index. |
Cost maths
Sparse is the cheap leg. Index RAM ≈ 30% of text: 10M passages × 1.5 KB = 15 GB text → ~5 GB index, fits a single 32 GB node. Assume ~$0.43/hr for a 64 GB memory-optimised cloud VM (illustrative on-demand list price, Oct 2026) → ~$310/month per replica with no per-query model cost. Compare a dense leg: 10M × 768-d float32 = 30.7 GB of vectors before graph overhead, plus an embedding call per query.
Operating it
- Recall@20 on paraphrased golden queries < 0.6 while identifier queries are fine — add a dense leg
- p99 > 100 ms on long (20+ term) queries — cap query terms or switch the rewriter off for keyword traffic
- Shard size > 50 GB — split the index (see Sharding & partitioning)
- Zero-result rate per query class — alert if > 3% or doubles week-over-week
- Recall@10 on the identifier slice of the golden set — alert below 0.9
- Search p99 latency — alert above 100 ms
When it goes wrong
Support engineers reported that searching an exact error code in the internal wiki returned nothing, though the page existed.
A golden-set query class 'exact identifier' dropped from 0.95 to 0.40 recall@10 after a mapping change.
A new English stemming analyzer split ERR_QUOTA_EXCEEDED into err, quota, exceed and the stop-word filter dropped the short tokens; the exact token no longer existed in the index.
Added a keyword sub-field and a code-aware analyzer for identifier-like tokens, boosted it in the query, and added identifier queries to the release Golden set.
Related terms
Where it shows up
Sources
- The Probabilistic Relevance Framework: BM25 and Beyond — Robertson & Zaragoza, 2009paper
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models — Thakur et al., 2021paper
- Elasticsearch reference: Similarity module — Elastic, 2026docs
Check yourself
Three questions. Every option is explained once you answer.