How would you maintain embedding consistency when documents are updated?
- 1 min — define consistency
- 2 min — version stamps and content hashes
- 1 min — orphan deletion and resumable backfills
- 1 min — reconciliation as a dashboard metric
Consistency means: every live vector was produced by the current model version, from the current document version, with the current chunking config. Enforce it by stamping each vector with {model_version, chunker_version, doc_version, content_hash} and treating any mismatch as a repair task.
Chunk-level content hashing: on a document update, re-chunk, hash each chunk, and only re-embed the ones whose hash changed. Delete vectors for chunks that no longer exist. This is also how you make backfills resumable — the hash tells you what's already done.
Run a reconciliation job that scans for drift — vectors with stale version stamps, orphaned vectors whose source document is gone, documents with no vectors at all. Report it as a number on a dashboard. Indexes rot silently; the only defence is a job that looks. Also make the whole pipeline idempotent so a replay fixes rather than duplicates. Org scope: the reconciliation job and its dashboard are owned by the platform team with a paging threshold; content teams get a weekly report for their sources only.
What the answer sounds like at each level
Re-embed a document whenever it changes.
Stamps vectors with model, chunker and doc versions plus content hash, re-embeds only changed chunks and deletes orphans.
Runs continuous reconciliation with drift metrics and alert thresholds, and makes the whole pipeline idempotent so replay repairs instead of duplicates.
Follow-ups the interviewer will push
1Now reconciliation finds 2% of vectors with a stale chunker version.
That is a partial rollout of a chunker change. Queue those documents for re-chunk and re-embed at low priority, and add a gate: no new chunker version goes live until the backfill finishes or the old version stays readable.
2Now the vector store doesn't support transactions across delete and insert.
Write new chunk vectors first with the new doc_version, then delete the old version; filter reads to the latest doc_version from the registry so a half-finished update never shows duplicates.
Red flags interviewers listen for
- No version stamps on vectors.
- Re-embeds whole documents on every edit.
- No job that looks for orphans or missing vectors.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Definition | States model, chunker, doc version and hash as the consistency key. |
| Efficiency | Chunk-level diffing with orphan cleanup. |
| Operability | Reconciliation with alert thresholds and idempotent replay. |
Sources
- Retrieval-Augmented Generation for Knowledge-Intensive NLP Tasks — Lewis et al., 2020paper
- Efficient and robust approximate nearest neighbor search using Hierarchical Navigable Small World graphs — Malkov & Yashunin, 2016paper