Compaction & soft delete
Background rewriting of an index's segments to drop deleted and overwritten entries, reclaiming space and restoring search speed and recall.
Most search and vector stores don't erase data immediately when you delete or update it. They mark the old entry as dead (a Tombstone) and write the new one elsewhere. Over time the index fills with dead entries that waste space and slow searches. Compaction is the background job that rewrites the index without them. It is like defragmenting: invisible when it works, painful when it falls behind.
A notebook where you cross out lines instead of erasing them. Eventually you copy the live lines into a fresh notebook so it is readable again.
Running force-merge on a live, write-heavy index during business hours. It saturates disk I/O and creates huge segments that are expensive to merge again later.
Log-structured stores (LSM trees, Lucene segments) are append-only: updates are delete + insert, deletes set a bit in a live-docs bitmap, and merges rewrite segments to drop them. Elasticsearch/OpenSearch merge continuously under a tiered policy; _forcemerge?only_expunge_deletes=true forces it but is heavy I/O, so only run it on indexes no longer receiving writes. HNSW graphs suffer extra: deleted nodes usually stay in the graph as routing points, so heavy churn degrades ANN recall (efSearch, nprobe) and wastes memory until a rebuild. Qdrant's optimizer vacuums a segment when its deleted fraction passes a threshold (default 0.2); Milvus runs compaction to merge small segments and purge deletes; pgvector relies on Postgres VACUUM (autovacuum), which can lag on large, write-heavy tables. Compaction competes with queries for CPU and disk, so schedule heavy jobs off-peak and throttle them. For workloads that rewrite most of the corpus regularly (daily re-embedding), building a fresh index and swapping an Index alias is cheaper than compacting.
Leave engine optimizers on (Qdrant deleted_threshold 0.2, Milvus auto-compaction, Elasticsearch tiered merge); per-table autovacuum tuning for pgvector (vacuum_scale_factor 0.02–0.05); heavy force-merge or REINDEX only off-peak and only on cold indexes; rebuild + alias swap when weekly churn >30%.
A customer-support knowledge base of 4M chunks re-ingests updated articles nightly; ~6% of chunks change per day. After three weeks without vacuum on a pgvector table, dead tuples reach 38% of the table, the HNSW index is 1.6× its live size, and p95 vector query latency rises from 35 ms to 90 ms with recall@10 down 3 points. Tuning autovacuum (autovacuum_vacuum_scale_factor=0.02 on that table) and a weekly REINDEX CONCURRENTLY brings both back.
Updates in vector stores are delete + insert, so I budget for compaction from the start: track deleted-ratio per index, let the engine's optimizer vacuum at ~20%, and for corpora that churn more than ~30% a week I rebuild and swap an alias rather than compact. Expect: 'why did recall drop with no code change?' Tombstoned HNSW nodes accumulating; check the deleted ratio before tuning ef_search.
Numbers worth knowing
- Qdrant vacuum trigger
- 20% deleteddefault optimizer `deleted_threshold` per segment
- Postgres default autovacuum trigger
- 20% of rows + 50too lazy for large, write-heavy embedding tables
- Latency growth from dead entries
- ~1.5–3×observed when dead fraction passes ~30–40% on HNSW indexes
How to decide
| When | Do this |
|---|---|
| Churn <10% per week | Default engine compaction; just monitor the deleted ratio. |
| Churn 10–30% per week | Tune thresholds tighter and schedule off-peak compaction. |
| Churn >30% per week or full re-embed | Build a fresh index and swap the Index alias. |
| Index is read-only (archive) | Force-merge once to minimal segments, then leave it. |
Cost maths
Dead space costs RAM for HNSW: 14 GB of dead vectors on memory-resident nodes at ~$5–8/GB-month (memory-optimised cloud instances, illustrative Oct 2026) ≈ $70–110/month, plus the latency hit. A compaction pass costs a few CPU-hours and I/O off-peak. A full rebuild of 4M chunks at 300 tokens each = 1.2B tokens ≈ $24 at $0.02/M, cheaper than you think when churn is high.
Operating it
- Deleted ratio >25% on any segment/index for >24 h
- Compaction jobs overlapping business hours because they no longer finish overnight
- p95 vector latency up >50% with flat traffic
- Deleted/total ratio per index or segment: alert >25%
- Dead tuples on pgvector tables (
pg_stat_user_tables.n_dead_tup): alert >20% of live - Golden-set recall@10 weekly: alert on a drop >2 points with no deploy
When it goes wrong
Answer quality on a developer-docs assistant slowly degraded over two months; no deploys correlated.
Weekly golden-set eval showed recall@10 down from 0.89 to 0.81; index stats showed 41% deleted documents.
Nightly re-ingest rewrote every chunk (non-deterministic chunk ids), so each night added a full copy and tombstoned the old one; compaction couldn't keep up.
Content-hash chunk ids so unchanged chunks are skipped, one rebuild + alias swap, and an alert on deleted ratio.
Where it shows up
Sources
- The log-structured merge-tree (LSM-tree) — O'Neil et al., 1996paper
- Efficient and robust approximate nearest neighbor search using HNSW graphs — Malkov & Yashunin, 2016paper
Check yourself
Three questions. Every option is explained once you answer.