Attribution & citations
Linking each claim in an answer to the exact source passage that supports it, and measuring whether those citations are correct and complete.
Attribution means the answer shows where each statement came from, like footnotes. Good attribution lets a reader click and check, and lets you measure two things automatically: does the cited passage really support the sentence (citation precision), and does every factual sentence have a citation (citation recall).
A news article where every quote names who said it and when, so an editor can call the source to confirm.
Counting 'has citations' as success. A model will happily append [S2] to a sentence S2 does not support; only a support check catches that.
Two implementation styles. Inline ids: chunks are labelled S1…Sn and the model writes [S3] after sentences; cheap, but the model can cite the wrong id. Span-level: the API returns the exact character range of quoted evidence (Anthropic Citations returns cited_text with document and character or page indices), which makes verification and UI highlighting trivial. Measurement follows ALCE (Gao et al. 2023): citation recall = share of sentences fully supported by their cited passages; citation precision = share of individual citations that are relevant to their sentence. Both use an NLI model or LLM judge. Rashkin et al.'s AIS framework defines the human standard: would a reader agree 'according to this source, the claim is true'? Attribution is not Faithfulness by another name: an answer can be faithful to the context but cite the wrong chunk, or cite correctly for some claims and invent others.
Native span citations (Anthropic Citations, or the provider equivalent) when available; else XML-tagged chunks with short ids and an instruction to cite after every factual sentence. Nightly job scores citation precision and recall on 200 sampled answers with an LLM judge; UI renders citations as clickable highlights.
Financial-filings Q&A: 'How much did the company spend on share buybacks in FY2025, and how does that compare to FY2024?'. Answer: 'FY2025 repurchases were $4.2B [S1], up from $3.1B in FY2024 [S1], under a $10B authorisation approved in March 2025 [S3].' The judge finds S1 supports both numbers and S3 supports the authorisation, but S3 actually says February. Citation precision 3/3 relevant, citation recall 2/3 sentences supported: the date is flagged and the UI shows it unhighlighted.
Say: I use span-level citations where the provider supports them, otherwise inline ids, and I track ALCE-style citation precision and recall on a sample with a judge calibrated against 100 human labels. Expect: 'citations look right but users still complain'. Check citation recall per sentence type; numbers and dates fail most often.
| Answer sentence | Cited | Supported? |
|---|---|---|
| FY2025 buybacks $4.2B | S1 | Yes |
| FY2024 buybacks $3.1B | S1 | Yes |
| $10B authorised March 2025 | S3 | No: says Feb |
| Citation recall | 2 / 3 = 0.67 |
Numbers worth knowing
- Citation recall target
- ≥ 0.9customer-facing; measured per sentence on a sample
- Judge sample size
- 200 answers/daygives ±4-point 95% CI at p ≈ 0.85
- Judge-human agreement to trust it
- ≥ 0.8 (Cohen's κ ≥ 0.6)on 100 human-labelled sentence-citation pairs
How to decide
| When | Do this |
|---|---|
| Provider offers span citations | Use them; they make verification and highlighting cheap |
| Answers mix sourced facts with general reasoning | Require citations only on factual sentences; tag reasoning sentences |
| Citation recall < 0.8 on numbers and dates | Extract numbers via structured output from the cited span, then template the sentence |
Cost maths
Judge scoring at 200 answers/day × ~3k tokens each on a small model at $0.80/M input (illustrative, Oct 2026) ≈ $0.48/day, under $15/month. Human calibration: 100 pairs × 1 min ≈ 2 hours per quarter.
Operating it
- Citation precision < 0.85 for two consecutive days
- New document type (scanned PDFs, tables) added to the corpus: re-calibrate the judge
- Users click citations on > 20% of answers: invest in span-level highlighting
- Citation precision and recall on daily sample; alert < 0.85
- Share of factual sentences without a citation; alert > 5%
- Citation click-through and 'source didn't help' feedback rate; trend
When it goes wrong
Research-assistant answers cited correct papers but users found the quoted results were not in them
Support tickets, then a sampled audit: citation precision 0.71 while 'has citation' was 99%
Chunks were labelled by document, not by passage; the model cited the right paper for a number that came from another retrieved paper
Passage-level ids and span citations; judge checks the specific span; numbers without a supporting span are removed
Where it shows up
Sources
- Enabling Large Language Models to Generate Text with Citations (ALCE) — Gao et al., 2023paper
- Measuring Attribution in Natural Language Generation Models — Rashkin et al., 2021paper
- Citations — Anthropic, 2025docs
Check yourself
Three questions. Every option is explained once you answer.