How do you prevent outdated documents from influencing answers?
- 1 min — freshness as metadata
- 2 min — hard filter vs soft boost, tombstones
- 1 min — historical queries
- 1 min — undeclared staleness and owner reports
Treat freshness as a first-class field, not an afterthought. Store effective_date, valid_until, version and status as metadata. Filter at query time (hard-exclude superseded), boost recency in ranking (soft), and run a reaper job that detects source deletions. Critically, deletion in the source must propagate to the index — tombstones, not just absence.
Policy and pricing content. We kept superseded versions in the index — removing them broke "what was the policy in March?" — but excluded them by default and only included them when the query carried a historical intent.
The hard part is detecting staleness you weren't told about: a Confluence page nobody deleted but everyone stopped trusting. Signals: last-edited date, page views, whether anyone clicked the citation. We shipped a weekly staleness report to content owners, which solved more quality problems than any retrieval change. Org scope: freshness metadata is a contract between content owners and the platform; publish the required fields (effective_date, status, owner) and reject ingestion of documents without an owner.
What the answer sounds like at each level
Delete old documents from the index.
Stores version and validity metadata, hard-filters superseded docs, soft-boosts recency and propagates deletions with tombstones.
Handles staleness nobody declared using usage and edit signals, and builds a reporting loop to content owners.
Follow-ups the interviewer will push
1Now a user asks a historical question: 'What was the claims policy in March 2025?'
Classify temporal intent, then filter on effective_date <= 2025-03-31 AND (valid_until IS NULL OR valid_until >= 2025-03-01) instead of the default 'current only' filter.
2Now the source system deletes a document but the delete event is lost.
A nightly reconciliation diffs the registry against the source's document list and tombstones orphans. Alert if the orphan count exceeds ~0.1% of the corpus — that means the event path is broken.
Red flags interviewers listen for
- Relies on recency boosting alone, letting superseded docs still rank.
- No plan for deletes that happen in the source.
- Destroys history, breaking point-in-time questions.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Data model | Names concrete validity and version fields. |
| Correctness | Propagates deletions and handles point-in-time queries. |
| Operability | Detects silent staleness and closes the loop with owners. |
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020paper