Backfill
Re-processing the whole corpus into a new index (new model, chunker or schema) alongside the live one, then switching traffic once it is verified.
When you change something fundamental, such as the Embedding model, the chunk size or the Chunk metadata schema, every stored chunk is out of date. A backfill rebuilds the whole index the new way.
It is done next to the live index, not on top of it, so users keep getting answers while the new one is built, checked against test questions and then switched on in one step.
Building a new bridge beside the old one, load-testing it, then moving traffic across in one go, instead of replacing the deck while cars are on it.
Re-embedding in place into the live index. For hours or days the index mixes two models' vectors, results are silently wrong, and there is no clean rollback.
Pattern: create index v2 → bulk-process from the Document registry and raw store (not from the source APIs) → Dual-write or replay live changes into both v1 and v2 during the build (or record the CDC position at backfill start and replay from it) → verify v2 → switch an Index alias atomically → keep v1 for rollback → delete v1 after a soak period.
Verification: document and chunk counts vs the registry, golden-set recall@k and nDCG vs v1, latency at production QPS, and Shadow traffic comparing v1 and v2 results on live queries (overlap@10, answer diffs). Throughput planning: embedding rate × chunks, plus provider rate limits (tokens per minute) or GPU count; use batch APIs at ~50% discount when hours of latency are fine. Make it resumable: process in shards keyed by doc id, checkpoint per shard, idempotent upserts. Mixing vectors from two models in one index is never valid; the switch is all-or-nothing per index.
Versioned indexes behind an alias (docs → docs_v7); backfill from the registry's cached text, sharded by doc-id hash with per-shard checkpoints; provider Batch API when latency allows; record CDC/stream position at start and replay; gate the switch on count parity (±0.1%), golden-set recall no worse than −0.5 points, and 2–3 days of shadow traffic; keep v1 for 7–14 days.
Code search across 9,000 repositories, 48M chunks, migrating from a 768-dim model to voyage-code-3 at 1024 dims. Build code_v2 from the registry's cached chunk text: 48M × 220 tokens = 10.6B tokens. Throughput is capped at ~3M tokens/min by the provider tier, so ~2.5 days. CDC position recorded at start; 1.1M changes replayed into v2 after. Shadow traffic for 3 days: overlap@10 with v1 = 0.58, golden-set recall@10 0.74 → 0.83, p95 latency 28 → 31 ms. Alias code_search switched; v1 kept 14 days, then dropped.
Describe it as a deployment: "New index from the registry, replay changes from a recorded CDC position, verify counts, golden set and shadow traffic, alias swap, v1 retained for rollback." Expect: "how long and how much for 100M chunks?" Do the token arithmetic, divide by the rate limit or GPU throughput, double it for retries, and state the RAM to run v1 and v2 together.
Numbers worth knowing
- Duration
- tokens ÷ throughput, ×1.5–2 for retriese.g. 10.6B tokens at 3M tokens/min ≈ 2.5 days
- Batch API discount
- ~50%OpenAI and others, up to 24 h turnaround (Oct 2026)
- Extra RAM during switch
- ~2× index sizev1 and v2 both live until v1 is dropped
How to decide
| When | Do this |
|---|---|
| Embedding model, dimension, prefix or chunker changes | Full backfill into a new index; never in place. |
| Only metadata fields are added | Payload update in place; no re-embed. |
| Backfill would exceed a week at current limits | Raise rate limits, self-host on GPUs for the backfill, or use the Batch API. |
Cost maths
Cost = chunks × tokens × price + 2× index RAM for the overlap window. 48M × 220 = 10.6B tokens × $0.18/M (voyage-code-3 list price, illustrative, Oct 2026) ≈ $1,900, or ≈ $950 via a 50% batch discount where offered. Running v1 + v2 (2 × 60 GB) for 14 days at ~$5/GB-month ≈ $280. Self-hosted alternative: 48M chunks at ~800 chunks/s per L4 ≈ 17 GPU-hours ≈ $15 plus engineering time.
Operating it
- Backfill duration exceeds a week: move to batch APIs or dedicated GPUs
- Backfills more than quarterly: invest in resumable, sharded backfill tooling
- Corpus >100M chunks: backfill per tenant or shard with staged alias switches
- Backfill progress (shards done, tokens/min, error rate): alert on stalls >30 min
- Count parity v1 vs v2 vs registry before switch: block switch if difference >0.1%
- Post-switch golden-set recall and shadow overlap: auto-rollback alias on a drop >2 points
When it goes wrong
A week after an embedding upgrade, recall in a support assistant was fine overall but 40% of new articles never appeared.
Freshness probe: articles published in the backfill window were missing from the new index.
The backfill read a registry snapshot taken at start; live changes during the 4-day build went only to v1 and were never replayed into v2.
Record the CDC position at backfill start and replay to v2 before switching; add a pre-switch check comparing per-day document counts between v1 and v2.
Where it shows up
Sources
- Elasticsearch aliases — Elastic, 2025docs
- Debezium documentation — Debezium, 2025docs
Check yourself
Three questions. Every option is explained once you answer.