Contextual enrichment (breadcrumbs)
Prepending a short LLM-written note situating each chunk in its document before embedding and indexing. Anthropic measured 35–67% fewer retrieval failures.
A chunk cut from a long document often makes no sense alone: 'Revenue grew 3% over the prior quarter' does not say which company or which quarter. Contextual enrichment asks a language model to write one or two sentences explaining where the chunk comes from, and glues that note onto the front of the chunk before it is stored and searched.
The chunk now carries its own context, so both meaning-based and keyword search find it more reliably.
Writing 'Q2 2025 Acme earnings call, CFO remarks' on the back of a photocopied page before filing it, so it still makes sense when pulled out of the folder.
Running enrichment without caching, paying full price to re-read the same 10k-token document for every one of its 30 chunks, or regenerating the whole corpus on every pipeline run instead of only changed documents.
Anthropic's Contextual Retrieval (Sept 2024): for each chunk, prompt a small model with the whole document plus the chunk and ask for 50–100 tokens of situating context; prepend it to the chunk for both the embedding and the BM25 index. On their benchmark (top-20 retrieval failure rate), contextual embeddings cut failures 35% (5.7% → 3.7%), adding contextual BM25 49% (→ 2.9%), and adding a reranker over the top 150 67% (→ 1.9%).
The cost is one LLM call per chunk, each reading the full document. Prompt caching makes that affordable: write the document to cache once, then each chunk call reads it at ~0.1× price. Anthropic reported $1.02 per million document tokens with Claude 3 Haiku. Simpler deterministic enrichment captures part of the gain at zero LLM cost: prepend the title, section breadcrumb and date from Chunk metadata. Late chunking is the embedding-side alternative. Treat the generated context as derived data: store the prompt version, regenerate on document change, and never let it introduce facts not in the document.
Step 1, always: prepend title › section path › date to every chunk. Step 2, if the golden set still shows context misses: Claude Haiku 4.5 (or another small model) with the document in a cached system block, ≤100-token context per chunk, temperature 0, prompt version stored on the chunk, regenerate only when the document's Content hashing changes. Apply to both dense and BM25 indexes.
Insurance policy wordings, 80k documents averaging 9k tokens, 300-token chunks. Raw chunk: 'This exclusion does not apply if the vehicle was used for commuting.'
Generated context: 'From the 2025 Private Motor Policy (UK), Section 4 Exclusions, clause 4.3 on business use.' Recall@20 on 400 broker questions: 0.83 → 0.90 with contextual embeddings + BM25. Generation with Claude Haiku 4.5 and caching cost about $3.80 per million document tokens: ~$2,700 for the 720M-token corpus, once.
Lead with the cheap version: "First I prepend title, section path and date deterministically, which is free. If misses are still about missing context, I run contextual retrieval with a small model and prompt caching, store the generated text with a prompt version, and regenerate only changed documents via the content hash." Expect: "what does it cost at 100M chunks?" Work through the cache-write and cache-read multipliers aloud.
Numbers worth knowing
- Retrieval failure reduction
- 35% (embed) · 49% (+BM25) · 67% (+rerank)Anthropic benchmark, top-20, Sept 2024
- Generation cost
- $1.02 per M document tokensClaude 3 Haiku with prompt caching, 800-token chunks, 8k docs (Anthropic)
- Context length per chunk
- 50–100 tokensadds ~15–30% to a 300-token chunk
How to decide
| When | Do this |
|---|---|
| Chunks reference unnamed entities ('the Company', 'this clause', 'Q2') | Contextual enrichment; deterministic breadcrumbs first. |
| Corpus changes heavily every day | Deterministic enrichment only, or enrich incrementally by content hash. |
| Documents exceed the model's context window | Enrich using the section, not the whole document. |
Cost maths
Per document with n chunks: cache write D × 1.25 + (n − 1) × D × 0.1 cached reads + n × (chunk + instructions) uncached + n × 80 output. Assume Claude Haiku 4.5 at $1/M input, $5/M output (illustrative list price, Oct 2026). D = 9k, n = 30, chunk + instructions = 350: 11.25k + 26.1k + 10.5k = 47.9k input-equivalent ≈ $0.048, plus 2.4k output ≈ $0.012. Total ≈ $0.06 per document ≈ $6.70 per M document tokens; Batch API halves it. 80k such documents ≈ $4,800 once.
Operating it
- Golden-set misses where the right chunk lacks the entity or period named in the query: >15% of misses
- BM25 recall low on queries using document titles or company names
- Cache hit rate on enrichment calls: alert below 90% (cost will spike)
- Share of generated contexts failing the entity-presence check: alert above 1%
- Enrichment lag behind ingestion: alert above 1 hour
When it goes wrong
Enriched chunks in a pharma regulatory corpus began citing the wrong product name.
Faithfulness checks flagged answers naming a drug that appeared nowhere in the cited chunk's source.
Multi-product submission PDFs; the context prompt saw the whole file and attributed sections to the first product mentioned.
Enrich per section with section heading and product field from metadata, add a validator rejecting contexts that mention entities absent from the section, regenerate affected chunks.
Where it shows up
Sources
- Introducing Contextual Retrieval — Anthropic, 2024blog
- Late Chunking: Contextual Chunk Embeddings Using Long-Context Embedding Models — Günther et al., 2024paper
Check yourself
Three questions. Every option is explained once you answer.