Replication
Keeping copies of each shard on several nodes for availability and read throughput. Replicas scale QPS; they don't fix an index that's too big.
Replication keeps extra copies of the data on different machines. If one machine fails, another copy keeps answering, and because every copy can answer searches, more copies means more searches per second. The catch is keeping copies in sync: right after a write, one copy may briefly have the new data while another doesn't yet.
A popular restaurant opening identical branches: more customers can be served at once, and if one branch closes, the others stay open — but a menu change takes a moment to reach every branch.
Treating replicas as backups. A mistaken delete_by_query, a corrupt re-embed or a wrong model version propagates to every replica within seconds. Keep snapshots (Elasticsearch snapshot to S3, Qdrant snapshots, Postgres PITR) and test restores.
Each shard has a primary (or leader) and replicas. Writes go to the primary and are copied: synchronously (Elasticsearch waits for in-sync replicas before acknowledging; Qdrant write_consistency_factor), or asynchronously (Postgres streaming replicas, where a read replica may lag by ms to seconds). Reads can go to any replica, so read QPS scales ~linearly with replica count until the coordinator or network saturates. Vector-specific wrinkles: each replica builds its own ANN graph (Elasticsearch/OpenSearch replicas index independently, so a 1-hour HNSW build happens on every copy — segment replication in OpenSearch avoids this by copying segments), and HNSW graphs on different replicas aren't identical, so the same query can return slightly different top-k depending on which replica answers (pin a session to a replica with preference if consistency matters). Place replicas across availability zones; 2 copies tolerate one node loss, 3 tolerate one loss during maintenance. Replicas are not backups: a bad bulk delete replicates instantly.
1 replica minimum in production (2 copies), across availability zones; 2 replicas for indices that must survive a node loss during maintenance. Replica count sized from peak QPS ÷ per-copy QPS at your p95 SLO. Daily snapshots to object storage with a quarterly restore drill.
Customer-support search, 60M chunks in 8 shards, 1 replica (16 shard copies on 8 nodes across 2 AZs). Peak 1,200 QPS, each shard copy sustains ~250 QPS at p95 < 30 ms. Required copies per shard = 1,200 / 250 ≈ 5 → bump to 4 replicas (5 copies), 40 shard copies on 20 nodes. Writes of 300 docs/s now incur 5× indexing CPU — so they enable segment replication to index once and copy segments.
Separate the axes and the failure modes: "Shards for size, replicas for QPS and availability, backups for correctness. With vectors I care that every replica rebuilds the graph, so I either budget 2–3× indexing CPU or use segment replication." Expect: what about read-your-writes? Route a user's reads to the primary or pin the session for N seconds after their write.
Numbers worth knowing
- Read scaling
- ~linear in copiesuntil coordinator/network saturates
- Indexing CPU
- × number of copiesdocument replication; ~1× with segment replication
- Async replica lag
- ms to secondsPostgres streaming replication under normal load; minutes under heavy writes
How to decide
| When | Do this |
|---|---|
| QPS above what one copy serves at p95 SLO | Add replicas: copies = peak QPS ÷ per-copy QPS (+1 for failure headroom). |
| Heavy re-embedding or bulk writes | Segment replication or reduce replicas to 0 during bulk load, then restore. |
| Users must see their own writes immediately | Route their reads to the primary for a short window or use session preference. |
Cost maths
Cost scales with copies: N copies × index RAM. 60M × 1024-d int8 + graph ≈ 75 GB per copy; 5 copies = 375 GB ≈ 8 × 64 GB nodes at ~$0.43/hr (illustrative Oct 2026) ≈ $2,500/month. A Semantic cache serving 20% of queries cuts needed copies from 5 to 4, saving ~$500/month.
Operating it
- Per-copy QPS > 70% of measured capacity at peak — add a replica
- Replica lag > freshness SLO (e.g. 30 s) — reduce write load or move to sync replication
- Single-AZ placement of all copies of any shard — fix allocation awareness immediately
- Unassigned or out-of-sync replica shards — page on any for > 5 min
- Replica lag (async systems) — alert above freshness SLO
- Doc count change rate — alert on > 5% drop in 10 min
- Snapshot age and last successful restore drill — alert if > 36 h / > 90 days
When it goes wrong
A bulk cleanup job meant to delete one tenant's test data removed 18% of production chunks across all tenants; answers degraded within a minute.
Index document-count alert (−18% in 5 min) and a spike in zero-result queries.
A malformed filter in delete_by_query matched far more documents than intended; replication faithfully copied the deletes to every replica.
Restored from the nightly snapshot plus a replay of the last 14 hours of ingestion from the source-of-truth queue; added dry-run counts and a max-delete-fraction guard to bulk jobs.
Related terms
Sources
- Elasticsearch reference: Scalability and resilience — Elastic, 2026docs
- PostgreSQL documentation: Log-shipping standby servers — PostgreSQL Global Development Group, 2026docs
- Qdrant documentation: Distributed deployment — Qdrant, 2026docs
Check yourself
Three questions. Every option is explained once you answer.