When is a reasoning model worth paying for?
- 2 min — how they're trained (RLVR, GRPO) and what they buy
- 2 min — cost and latency math with thinking tokens
- 2 min — routing design and budget curve
- 1 min — ownership, rollout, fallback
Reasoning models generate hidden thinking tokens before the answer. They are trained with reinforcement learning from verifiable rewards (RLVR): sample many solutions to maths and code problems, score each with a checker (unit tests pass, answer matches), and update the policy toward the higher-scoring ones. GRPO, used for DeepSeek-R1, samples a group of e.g. 16 answers per prompt and uses each answer's reward relative to the group mean as its advantage — no separate value model. The result trades latency and output tokens for accuracy on multi-step search: planning, debugging, multi-hop analysis, hard maths. On lookup, classification or rewriting they add cost and latency without measurable gain.
A financial-filings analytics assistant gets two kinds of question. "What was Q2 revenue?" — single-table SQL against a known schema, a fast model answers in 1.5 s. "How did segment margin change after the acquisition, excluding one-offs?" — multi-table joins and definitional choices; a reasoning model with an 8k thinking budget answers in 25 s. A small intent classifier routes by difficulty.
- Treat thinking budget as a dial: plot accuracy vs budget (1k / 4k / 16k) on your own golden set; in most task sets the curve flattens and the last doubling buys <2 points.
- Hidden tokens are billed as output: 2,000 thinking tokens at $15 / M add $0.03 to a request that otherwise costs $0.015 — 3× the cost before any quality gain. They also dominate p95 latency.
- In agent loops, use the reasoning model for planning and a cheaper model for routine tool steps.
- Temperature is usually fixed on these models; tune budget and prompt, not sampling.
- Org scope: the platform team exposes reasoning tiers through the gateway with per-route budgets and a cost dashboard; product teams must show an eval delta to get a route onto the expensive tier. Rollout: route 5% → 25% → 100% of hard-class traffic with a latency SLO guard.
- Where it breaks: defaulting every call to the strongest tier. Cost multiplies, p95 breaks the SLO, and easy traffic gets no better.
assume $3 / M input and $15 / M output tokens (illustrative frontier list prices, Oct 2026); cache write 1.25× input for a 5-minute TTL, cache read 0.1×. Base request 3,000 in / 400 out = $0.009 + $0.006 = $0.015 ($15 per 1k). Add 2,000 thinking tokens = +$0.030 → $0.045 ($45 per 1k). Routed at 20% hard: 0.8 × $0.015 + 0.2 × $0.045 = $0.021 ($21 per 1k), vs $45 per 1k all-reasoning.
| Route | Tokens (in / think / out) | $ per 1k requests |
|---|---|---|
| Fast model | 3,000 / 0 / 400 | $15 |
| Reasoning | 3,000 / 2,000 / 400 | $45 |
| Routed 80/20 | blended | $21 |
What the answer sounds like at each level
Knows reasoning models think before answering and cost more; uses them for hard problems.
Measures accuracy vs thinking budget on a golden set, prices hidden tokens, and routes by difficulty.
Explains RLVR/GRPO, runs reasoning tiers as a gated platform capability with per-route budgets, SLO guards and misroute metrics.
Follow-ups the interviewer will push
1How are reasoning models trained differently?
RL from verifiable rewards: sample many solutions, score with checkers (tests, exact answers), and push the policy toward high-reward samples. GRPO normalises rewards within a group of samples per prompt instead of training a value model.
2Now the reasoning provider has an outage. What happens?
The gateway fails over hard-class traffic to the fast model with a 'lower confidence' flag, or queues non-urgent questions. The fallback must be in the eval suite so you know its accuracy on hard items (here 74%).
3Can you cap cost per request?
Yes: set max thinking tokens and max output tokens per route, a wall-clock timeout, and a per-tenant monthly budget enforced at the gateway.
Red flags interviewers listen for
- Routes everything to the strongest model 'to be safe'.
- Forgets thinking tokens are billed as output.
- Tunes temperature on a model that fixes it instead of tuning budget.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Understanding | Explains verifiable-reward RL and why it helps search-heavy tasks. |
| Economics | Works a per-request cost with thinking tokens and a blended routed cost. |
| Operational design | Router, budgets, SLO guard, fallback, misroute metric. |
Routing instead of upgrading
Illustrative exampleOur financial-analytics assistant (≈1.1M questions/month) had been moved wholesale to a reasoning model after two wrong answers reached an executive review. Wrong-answer reports fell, but blended cost went from $0.011 to $0.052 per question and p95 latency rose from 4 s to 31 s against a 10 s SLO.
I was the tech lead on a team of 4 engineers and 1 analyst. My job was to keep the quality gain and get back under the SLO and a $0.02 cost ceiling. The head of finance analytics opposed any rollback: "one more wrong number in a board pack costs more than the whole model bill."
I sampled 2,000 questions from one week and had the analyst label difficulty; 78% were single-table lookups. The team built a 450-item golden set (300 easy, 150 hard) with verified SQL results. I designed the routing experiment and wrote the router; two engineers built the eval harness and a fallback path; one handled gateway budgets. On hard items the reasoning model scored 91% vs 74% for the fast model; on easy items both scored 97%. We shipped an intent classifier routing only hard questions to the reasoning tier, with an 8k thinking cap and a 20 s timeout falling back to the fast model plus a "verify" banner. To win over the finance lead I showed her the 300 easy items side by side — identical answers — and gave her a weekly misroute report.
Hard-item accuracy stayed at 90% (n=150), easy at 97% (n=300); blended cost fell to $0.017 per question and p95 to 8.5 s. The router misrouted 3.2% of hard questions to the fast tier in the first month; we cut it to 1.4% by adding the 60 worst misroutes as training examples. What I'd do differently: build the misroute metric before launch — for two weeks we only learned about misroutes from user complaints.
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI, 2025paper
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models — Shao et al., 2024paper