Calculators
Do the maths, not the vibes
The three estimates that come up in every design review and staff interview. Change any input; the formulas are on the linked term pages.
KV-cache memory and concurrent users
Explained on KV cache.
Per token320 KB
Per user at this context10.49 GB
Concurrent users that fit8
Defaults: a 70B GQA model (80 layers, 8 KV heads, head dim 128) in BF16, with ~85 GB left after FP8 weights on 2×H100. Ignores activation and fragmentation overhead (paged attention keeps the latter near zero).
Cost per request, with prompt caching
Explained on Cost per query and Prompt caching.
Per 1,000 requests$10.05
Per month$30,164
Saved by caching33.0%
Illustrative list prices — check your provider. Assumes cache reads at 0.1× input and misses re-writing the prefix at 1.25× (a 5-minute TTL); batch APIs typically take ~50% off on top.
Agent cost per task
Explained on Agent cost & token budgets.
Input tokens processed249,000
Cost per task, no caching$0.815
Cost per task, prompt caching$0.243
Each turn re-sends the whole transcript. Illustrative prices; cache reads at 0.1×, writes at 1.25×.