SPLADE
Learned sparse retrieval: a BERT-style model outputs weighted vocabulary terms, including expansions, served from an ordinary inverted index.
SPLADE is a model that turns text into a weighted word list, like BM25, except the weights are learned by a neural network and it can add related words that never appeared in the text. A note about 'myocardial infarction' can also get weight on 'heart' and 'attack'. Because the output is still a sparse word list, it runs on the same fast inverted index as keyword search while fixing much of the synonym problem.
A librarian who, when filing a book about 'automobiles', also writes 'car' and 'vehicle' on the index card and crosses out filler words nobody searches for.
Assuming learned sparse is free. Every model upgrade is a full GPU re-encode and reindex, expanded postings make the index 2–3× larger than BM25 and queries 2–5× slower, and English-trained checkpoints degrade badly on other languages and on code.
SPLADE runs a BERT encoder and reads its masked-language-model head: for each input position, a logit for every one of ~30,522 WordPiece entries. It applies log(1 + ReLU(x)) and max-pools over positions (SPLADE v2), giving one weight per vocabulary term. A FLOPS regulariser during training pushes most weights to zero so a passage keeps ~100–300 non-zero terms and posting lists stay short. Document encoding happens offline on GPU; query encoding either runs the same model (adds 10–30 ms on GPU) or uses a doc-only/inference-free variant where the query is just tokenised (OpenSearch neural-sparse doc-only models work this way). Production options as of Oct 2026: naver/splade-v3 and SPLADE++ checkpoints, Elastic's ELSER (Elasticsearch sparse_vector), OpenSearch neural sparse, Qdrant and Vespa sparse vectors. Weakness: it inherits the tokenizer, so unusual identifiers split into sub-words and match worse than raw BM25 on a keyword field.
On Elasticsearch: ELSER into a sparse_vector field, fused with BM25 and a dense leg via the rrf retriever. On OpenSearch: a doc-only neural sparse model so queries need no inference. Self-hosted: naver/splade-v3 encoding on an L4/A10G with batch 64 and max length 256.
Clinical intake notes, 3M notes. Note text: 'pt c/o chest tightness radiating to L arm'. SPLADE output includes chest 2.3, tightness 1.8, arm 1.4 plus expansions pain 1.1, cardiac 0.9, heart 0.7. A clinician's query 'possible cardiac pain' now overlaps on cardiac and pain. Index: 3M × ~200 terms ≈ 600M postings — about 2.5× a BM25 index of the same notes. Encoding: ~1,000 passages/s on one L4 → 50 minutes for the corpus.
When the stack is already Elasticsearch/OpenSearch and the team cannot take on a vector database, propose learned sparse first: "SPLADE buys most of dense retrieval's synonym recall on the inverted index we already run, with explainable term weights." Expect the follow-up on cost: GPU re-encode of the whole corpus on every model change, and larger postings — quantify both.
Numbers worth knowing
- BEIR zero-shot nDCG@10
- ≈0.47–0.50 averageSPLADE v2 / SPLADE++ checkpoints vs ≈0.43 for BM25
- Index size
- 2–3× BM25~100–300 non-zero terms per passage
- Encoding throughput
- ~500–1,500 passages/sone L4 GPU, 256-token passages, FP16
How to decide
| When | Do this |
|---|---|
| Mature Elasticsearch/OpenSearch stack, English prose, synonym misses | Add learned sparse before introducing a separate vector database. |
| Multilingual or code-heavy corpus | Prefer a multilingual dense model plus BM25; English SPLADE checkpoints underperform. |
| Query latency budget < 30 ms end to end | Use a doc-only (inference-free query) variant. |
Cost maths
Encode cost = passages ÷ throughput × GPU price. 50M passages ÷ 1,000/s = 13.9 GPU-hours; assume an L4 at ~$0.80/hr (illustrative on-demand, Oct 2026) → ~$11 per full re-encode. The bigger cost is RAM/disk: a 50M-passage BM25 index of ~20 GB becomes ~50 GB, i.e. one more 64 GB data node (~$310/month at $0.43/hr).
Operating it
- Query p99 doubles after enabling expansion — prune query terms to top 32 or go doc-only
- Index > 2.5× BM25 size — raise the sparsity regulariser / prune low weights at index time
- Non-English traffic > 20% — evaluate a multilingual dense leg instead
- Non-zero terms per document (p50/p99) — alert if p99 > 2× the baseline after a model change
- Exact-identifier recall@10 — alert below 0.9
- Query p99 latency on the sparse leg — alert above 80 ms
When it goes wrong
After switching an e-commerce catalog from BM25 to ELSER-only, searches for exact model numbers like 'WH-1000XM5' returned the wrong headphone generation.
Add-to-cart rate from search dropped 9% in an A/B test; the exact-SKU query slice showed the largest drop.
The WordPiece tokenizer split the SKU into sub-words that also occur in sibling models, and the learned weights spread across them.
Kept BM25 on a keyword SKU field as a separate leg fused with Reciprocal Rank Fusion (RRF); learned sparse handled descriptive queries only.
Sources
- SPLADE: Sparse Lexical and Expansion Model for First Stage Ranking — Formal et al., 2021paper
- SPLADE v2: Sparse Lexical and Expansion Model for Information Retrieval — Formal et al., 2021paper
- BEIR: A Heterogeneous Benchmark for Zero-shot Evaluation of Information Retrieval Models — Thakur et al., 2021paper
Check yourself
Three questions. Every option is explained once you answer.