Agent cost & token budgets
What an agent run costs in tokens, money and time. It grows quadratically with turns unless cached, so manage it as a distribution per success.
Every step an AI agent takes is a call to an AI model, and you pay for every Token it reads and writes. Because the agent re-reads the whole growing conversation at each step, a 20-step run costs far more than 20 single questions.
Agent cost covers money, time and compute. It varies a lot from one task to the next, so you budget and monitor it as a range rather than assume one price.
A taxi on the meter in unknown traffic: you know the rate, not the fare, so you set a limit and watch the meter.
Estimating cost as steps × price of one call. History is re-sent every turn, so input grows each step and the long-tail runs dominate the bill.
Per-turn cost = input × p_in + output × p_out, where input = prefix (system + tools) + all prior turns. With a fixed prefix P and Δ tokens added per turn, cumulative input over T turns is T·P + Δ·T(T−1)/2 — quadratic in T. Multi-agent systems designs multiply this (Anthropic: ≈4× chat tokens for agents, ≈15× for multi-agent). Reasoning tokens bill as output. Wall-clock is sequential model latency plus tool latency.
Levers in order of payoff: Prompt caching with a moving breakpoint (Anthropic: 5-min writes 1.25× base input, 1-hour writes 2×, reads 0.1× on most models, lower on some 2026 models; minimum cacheable prefix 512–4,096 tokens depending on model), compaction and output trimming (Context engineering), fewer and better tools (Tool design (agent–computer interface)), routing easy steps to smaller models, parallel tool calls, batch APIs (~50% off) for offline runs, and early exits. Govern with per-run budgets (Stop conditions & step budgets), per-tenant quotas through an LLM gateway, and the right metric: cost per successful task at p50 and p95. See Cost per query for the non-agent baseline.
Prompt caching on the stable prefix and a moving breakpoint on history; tool outputs trimmed to ≤2–5k tokens; compaction at 70% of the window; per-run caps ($1–2, 300k tokens); per-tenant monthly quotas in the gateway; cheaper model for classification and extraction steps; batch API for offline agents; dashboards of cost per successful task p50/p95 by agent version.
Customer-support agent with an 8k-token prefix (instructions + 10 tools) and ≈1.5k tokens added per turn (tool result + model output), 300 output tokens per turn, 20 turns: 445k cumulative input. Uncached at $3 / $15 per M: $1.34 + $0.09 ≈ $1.43 per run. With a cache breakpoint on the latest turn: ≈ 36.5k tokens written at 1.25×, ≈ 408.5k read at 0.1× → $0.26 + $0.09 ≈ $0.35. At a 70% resolution rate, cost per resolved ticket is $0.50 cached vs $2.04 uncached.
Show the quadratic formula, then the order of levers: cache, trim, cap, route. Recommend budgeting on cost per successful task at p95 because the mean hides the runs that loop.
Expect: "finance wants a per-ticket price" — quote p50 and p95 cost per resolved ticket with the assumptions (prices, cache hit rate, turns), and a hard per-run cap that bounds the worst case.
Numbers worth knowing
- Cumulative input over T turns
- T·P + Δ·T(T−1)/2P = prefix, Δ = tokens added per turn, no compaction
- Anthropic cache pricing
- writes 1.25× (5 min) / 2× (1 h); reads 0.1×of base input price, most models, Oct 2026
- Agent vs chat tokens
- ≈4× (multi-agent ≈15×)Anthropic, June 2025
- Batch discount
- ~50%offline/async jobs on major providers
Try the numbers
Each turn re-sends the whole transcript. Illustrative prices; cache reads at 0.1×, writes at 1.25×.
How to decide
| When | Do this |
|---|---|
| Runs exceed ~5 turns within a few minutes | Turn on prompt caching with a moving breakpoint first. |
| Cost per success is high but cost per run is fine | Fix success rate (tools, context), not price. |
| Runs are not user-facing | Use batch APIs and smaller models for simple steps. |
Cost maths
Assume $3 / M input, $15 / M output (illustrative list prices, Oct 2026), P = 8k, Δ = 1.5k, 300 output tokens/turn, T = 20. Uncached input = 20×8k + 1.5k×190 = 445k → $1.335; output 6k → $0.09; total ≈ $1.43. Cached (5-min TTL, turns < 5 min apart): writes = 8k + 19×1.5k = 36.5k × 1.25 × $3/M = $0.137; reads = 408.5k × 0.1 × $3/M = $0.123; total ≈ $0.35 (≈ 4× cheaper overall, ≈ 5× on input). At 100k runs/month: $143k vs $35k. At 70% success: $2.04 vs $0.50 per resolved task.
Operating it
- Cost per successful task p95 > 3× p50
- Cache-read share of input < 70% on multi-turn agents
- One tenant > 25% of agent spend
- Monthly agent spend growing faster than successful tasks
- Cost per successful task p50/p95 by agent version (alert at +25%)
- Cache-read share of input tokens (alert < 70%)
- Per-run cap hits (alert > 2%)
- Spend per tenant vs quota (alert at 80%)
When it goes wrong
Agent spend rose 3.8× in a week while traffic and success rate were flat.
Daily cost-per-success dashboard; cache-read share had dropped from 85% to 4%.
A prompt change inserted the current timestamp at the top of the system prompt, so the prefix changed on every call and every request missed the cache.
Moved volatile values (time, user name) after the cached prefix, added a CI check that the prefix hash is stable across requests, and an alert on cache-read share below 70%.
Related terms
Where it shows up
Sources
- Prompt caching — Anthropic, 2026docs
- How we built our multi-agent research system — Anthropic, 2025blog
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog
Check yourself
Three questions. Every option is explained once you answer.