What is a knowledge cutoff, and how do you design around it?
- 1 min — what a cutoff is and why coverage thins before it
- 2 min — date/version injection and retrieval
- 2 min — conflict and contamination evaluation
- 1 min — freshness ownership
Parametric knowledge comes from training data collected up to a date. Anything after — new library versions, prices, regulations, your own data — is unknown, and coverage thins in the last months before the cutoff because the web hasn't written much about them yet. Models frequently misstate their own cutoff and answer questions about recent events with confident, stale facts. Frontier models ship 6–12 months after their cutoff and stay in production 12–24 months, so production traffic routinely asks about things 1–3 years newer than the model.
A developer-docs assistant recommends pages/ router patterns for a Next.js 16 project. Fix: read the user's lockfile, retrieve docs for those exact versions, inject Today is 2026-10-12; project uses next@16.x into the system prompt (~25 tokens), and instruct the model to prefer provided docs over memory. On a 150-question version-sensitive set, stale-API answers fall from 22% to 4%.
- Freshness is a retrieval or tool problem, not a training problem. Re-training to keep up costs weeks per cycle; retrieval updates in minutes.
- Inject the current date and relevant versions — cheapest fix in this whole handbook.
- Evaluate the conflict case explicitly: context says X, memory says Y. Models side with memory on a measurable share of items; track that rate per model.
- Watch contamination: public evals created before the cutoff may be in training data and overstate quality.
- Org scope: a docs/knowledge platform team owns freshness SLAs for indexed sources (e.g. new docs searchable within 1 hour); product teams own the version-detection logic. Buy search infra, build the freshness pipeline.
- Where it breaks: retrieved docs from mixed versions; the model blends them. Filter retrieval by version metadata.
What the answer sounds like at each level
Knows the model doesn't know recent facts and that RAG or search helps.
Injects date and versions, retrieves version-filtered docs, and measures stale-answer rate.
Owns freshness SLAs as a platform property, evaluates context-vs-memory conflicts per model, and plans for models living 2 years past cutoff.
Follow-ups the interviewer will push
1Now a regulated tenant: tax rules changed last month.
Never answer from memory. Retrieve from an authoritative, dated rules source; cite the effective date; refuse if no source newer than the change exists.
2Why not fine-tune on new docs?
Fine-tuning injects facts unreliably, can't be updated quickly, gives no attribution, and must be redone on every base-model upgrade.
Red flags interviewers listen for
- Proposes regular fine-tuning to stay current.
- Trusts the model's statement of its own cutoff.
- Doesn't test what happens when context contradicts memory.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Understanding | Explains cutoff, thin pre-cutoff coverage, and model lifecycle gap. |
| Mitigation | Date/version injection plus version-filtered retrieval with a measured effect. |
| Evaluation | Tests context–memory conflict and contamination explicitly. |
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020paper