What problems occur when a document is chunked without preserving context?
- 1 min — four failure modes
- 1 min — why lost scope is a liability
- 2 min — breadcrumbs → preambles → parent retrieval
- 1 min — cost per fix
Four recurring failures: dangling references ("as described above", "this policy" — with no antecedent in the chunk); lost scope (a chunk about an exception reads as the rule); orphaned attributes (a number with no label); and split answers (the question needs two halves that now live in separate chunks and rank separately).
The dangerous one is lost scope. A chunk saying "employees receive 25 days of leave" retrieved without its heading "Policy for Germany" produces a confidently wrong answer for a US employee. That's not a quality bug, it's a liability.
Fix with contextual enrichment: prepend the document title and section breadcrumb to every chunk, or generate a one-sentence situating preamble per chunk at ingestion time (costly but effective). Then parent-document retrieval for the split-answer case. Frame the choice by cost: breadcrumbs are nearly free and get you most of the win. Org scope: enrichment is an ingestion-platform feature with a per-source budget; the content owner decides whether the per-chunk LLM preamble is worth paying for their corpus.
What the answer sounds like at each level
Chunks lose meaning when cut from their surrounding text.
Names the four failures and fixes them with breadcrumbs, contextual preambles and parent retrieval.
Ranks fixes by cost per quality point, calls out lost scope as a liability risk, and measures with scope-sensitive test questions.
Follow-ups the interviewer will push
1What does contextual enrichment cost on 10M chunks?
Assume ~8k-token parent doc plus 100-token output per chunk. With prompt caching on the document prefix, the per-chunk cost is mostly cache reads: 10M × 8k × $0.30/M ≈ $24k, versus ~$240k uncached at $3/M, plus ~1B output tokens ≈ $15k at $15/M. Batch APIs roughly halve either figure. Breadcrumbs are free by comparison.
2Now the corpus is multi-jurisdiction contracts.
Put jurisdiction in structured metadata as well as the breadcrumb, and filter on it from the user's profile. Text prepending helps ranking; a filter guarantees scope.
Red flags interviewers listen for
- Treats lost scope as a minor quality issue.
- Jumps to LLM-generated preambles without considering free breadcrumbs.
- No test for scope-sensitive questions.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Diagnosis | Names dangling references, lost scope, orphaned attributes, split answers. |
| Cost ranking | Orders fixes by cost and expected gain. |
| Risk | Treats scope errors as legal/compliance exposure. |
Sources
- Introducing Contextual Retrieval — Anthropic, 2024blog
- Lost in the Middle: How Language Models Use Long Contexts — Liu et al., 2023paper