TTL & cache invalidation
Time to live: how long a cached or stored item is valid before it expires. Too long serves stale data; too short throws away savings.
A TTL (time to live) is an expiry timer attached to data. A Semantic cache entry might live 24 hours, a login session 1 hour, a temporary download link 15 minutes. When the timer runs out, the item is deleted or treated as stale and fetched again. Picking the TTL is a trade-off between freshness and cost: longer TTLs save money and time but risk serving outdated answers.
The use-by date on milk: it is safe to use until then, and after that you buy fresh rather than guess.
Using TTL as the freshness mechanism for data that changes on events. A 24-hour TTL means up to 24 hours of wrong answers after every change; invalidate on change and keep the TTL as a safety net.
Set TTL from how fast the source changes and how bad stale data is, not from habit. Cache TTL should be shorter than the source's update interval or paired with event-based invalidation (delete cache keys on Change data capture (CDC) change events), because TTL alone guarantees staleness up to the TTL. Add jitter (±10%) to TTLs set in bulk, or everything expires at once and stampedes the backend; use request coalescing so one miss refills a hot key. Expiry is not always instant: Redis expires lazily on access plus a sampled background sweep; DynamoDB TTL deletes in the background with a lag that can be days, so filter expired items in queries. Provider-side caches have their own TTLs: Anthropic prompt caching is 5 minutes by default (refreshed on each hit) or 1 hour at a higher write price, which decides whether a cache survives between user turns. TTLs also serve compliance: retention limits on logs and prompts are TTLs with legal weight, so enforce them in the store, not in a cron job someone can forget.
Redis with SET key value EX <ttl> and ±10% jitter; cache keys include corpus/prompt version; CDC-driven invalidation by tag; single-flight on misses; DynamoDB TTL with an expires_at filter in reads; retention TTLs enforced in the store (S3 lifecycle rules, index ILM).
A customer-support assistant caches answers keyed by normalised question + knowledge-base version. TTL 24 h ± 2 h jitter, plus invalidation: when an article changes, the CDC consumer deletes cache keys tagged with that article id. Hit rate 31%, saving ~$410/day at 120k questions/day. Before invalidation was added, a pricing-page change took up to 24 h to reach answers and generated 140 tickets in one afternoon.
I set TTL from the source's change rate and the cost of staleness, and I never rely on TTL alone for correctness: event-based invalidation handles freshness, TTL is the backstop. I'd add jitter and request coalescing to avoid stampedes. Expect: 'what TTL for a semantic cache?' Hours for slow-changing docs, keyed by corpus version so a re-index invalidates everything at once.
| Data | Default TTL | Plus |
|---|---|---|
| Semantic cache answer | 24 h ± 10% | invalidate on doc change |
| Anthropic prompt cache | 5 min / 1 h | refreshes on each hit |
| Presigned download URL | 15 min | re-sign on view |
| Raw prompt/completion logs | 30 days | enforced in store |
| Idempotency keys | ≥ queue retention | covers redelivery |
Numbers worth knowing
- Anthropic prompt cache TTL
- 5 min default, 1 h option1-hour writes cost 2× base input vs 1.25× for 5-minute; reads 0.1×
- Jitter on bulk TTLs
- ±10%spreads expiry so a refill doesn't stampede the backend
- Max staleness with TTL only
- = TTLwithout event-based invalidation
How to decide
| When | Do this |
|---|---|
| Source changes on known events | Invalidate on the event; TTL is only the backstop. |
| Source changes unpredictably and staleness is costly | Short TTL (minutes) or no cache. |
| Multi-turn chat with a long shared prefix | Prompt cache with 5-min TTL if turns are frequent; 1-hour if gaps exceed 5 min. |
| Data has a legal retention limit | Store-enforced TTL (lifecycle rules), audited. |
Cost maths
Semantic cache value = hit rate × requests × cost per miss − cache cost. 120k questions/day × 31% × $0.011 ≈ $410/day saved; a Redis node (~$150–300/month) is noise. Prompt cache: a 20k-token system prompt at $3/M base (illustrative Oct 2026): write at 1.25× = $0.075, each read at 0.1× = $0.006 vs $0.06 uncached. If users pause >5 min between turns, the 1-hour TTL (write 2× = $0.12) pays off after 2 extra hits.
Operating it
- Stale-answer complaints after doc updates: add event invalidation
- Cache miss storms at fixed times: add jitter and coalescing
- Prompt cache hit rate <50% in multi-turn flows: test the 1-hour TTL
- Cache hit rate by intent: alert on a drop >10 points after a deploy
- Age of served cache entries vs source last-modified: alert if any served entry is older than its source change
- Backend QPS spikes aligned with expiry times: alert on >3× baseline in 1 min
When it goes wrong
A retail assistant quoted last week's promotional price for 18 hours after a price change.
Customer complaints and a merchant escalation; cache keys showed the old answer with 6 h TTL remaining.
Semantic cache relied on a 24 h TTL; price changes didn't invalidate entries, and cache keys didn't include the catalog version.
Catalog version in the cache key, CDC-driven invalidation by product id, and TTL cut to 6 h for price-bearing intents.
Where it shows up
Sources
- EXPIRE — Redis, 2025docs
- RFC 9111: HTTP Caching — IETF, 2022spec
Check yourself
Three questions. Every option is explained once you answer.