Reasoning models & extended thinking
Models trained with RL to think at length before answering, trading latency and output tokens for accuracy on maths, code and multi-step work.
Reasoning models are LLMs trained to work through a problem internally, often for thousands of tokens, before giving the final answer. This 'thinking' is built-in Chain-of-thought.
They are much better at hard maths, coding, planning and multi-step analysis, but slower and more expensive per answer. Most APIs let you set an effort level or thinking budget, so you choose how hard the model works.
The difference between a quick answer at the door and someone who says 'let me sit down with this for ten minutes'. The second takes longer and is worth it only when the question is hard.
Using maximum thinking for everything. Latency and cost rise sharply while simple tasks gain nothing, and long reasoning can overthink easy questions into wrong answers.
Training recipe: large-scale RL with verifiable rewards (RLVR) on problems with checkable answers — maths results, unit tests, format checks. DeepSeek-R1 used GRPO (Group Relative Policy Optimization, from DeepSeekMath): sample a group of G answers per prompt, score each with the checker, and use (reward − group mean) / group std as the advantage, which removes PPO's value model. R1-Zero's AIME 2024 pass@1 rose from 15.6% to 71.0% during RL alone. Long traces with backtracking and self-checking emerge because they earn reward. This opened a second scaling axis: test-time compute.
Engineering facts as of Oct 2026: reasoning tokens are billed as output and count against max output tokens; the visible answer waits behind them, so Time to first token (TTFT) grows by seconds to minutes. Control is by effort level (OpenAI reasoning_effort; Anthropic adaptive thinking with an effort setting on newer models, budget_tokens ≥ 1,024 on older ones). Many reasoning models fix sampling: OpenAI's o-series reject non-default temperature/top_p, and DeepSeek recommends temperature 0.6 for R1. Providers return summarised or encrypted reasoning, not raw traces, and some multi-turn APIs require you to pass thinking blocks back unmodified.
Route by difficulty: non-thinking (or lowest effort) for extraction, classification and chat; medium effort for multi-step analysis; high only for the hardest class, verified on your eval. Set max output tokens to cover reasoning + answer (OpenAI recommends reserving ≥ 25,000 tokens when starting out), log reasoning-token usage per route, and leave temperature/top_p at the provider's required defaults.
Coding agent over a monorepo: simple renames go to a fast non-thinking model (~1 s, ~300 output tokens); failing-test investigations go to a reasoning model at medium effort (~25 s, ~6,000 reasoning + 500 visible tokens). On the team's 200-task eval, routed accuracy is 86% vs 88% for always-high-effort, at 31% of the cost. The 2-point gap sits in cross-package refactors, which the router now sends to high effort.
Answer 'when would you use a reasoning model?' with routing, effort levels and evals, not 'always'. Staff line: 'Reasoning models let me buy accuracy with test-time compute; a router decides which requests are worth the purchase, and I budget reasoning tokens as output.' Expect: 'how are they trained?' — RLVR with GRPO or PPO on checkable tasks — and 'what breaks?' — truncated answers when reasoning eats max tokens, and parameters like temperature being rejected.
Numbers worth knowing
- R1-Zero AIME 2024 pass@1
- 15.6% → 71.0%RL alone, DeepSeek-R1 paper
- Reasoning tokens
- 1k–30k per hard taskbilled as output tokens
- Minimum thinking budget
- 1,024 tokensAnthropic budget_tokens on models that still use it
- Recommended starting reserve
- ≥ 25,000 output tokensOpenAI reasoning guide
How to decide
| When | Do this |
|---|---|
| Task is extraction, lookup or short classification | Non-reasoning model or lowest effort. |
| Multi-step maths, code debugging, planning with constraints | Reasoning model at medium effort; raise only where the eval shows gains. |
| Latency SLO < 2 s for the first visible token | No thinking on that path; precompute or run reasoning asynchronously. |
Cost maths
Cost = input × $in + (reasoning + visible) × $out. Assume $3/M input, $15/M output (illustrative, Oct 2026). 2,000 input + 6,000 reasoning + 500 visible = $0.006 + $0.0975 = $0.1035 per request vs $0.0135 without thinking — 7.7×. At 100k requests/day that is $10,350 vs $1,350 per day; routing 20% to thinking gives $3,150/day.
Operating it
- Reasoning tokens > 70% of total spend on a route → tune effort per task class
- Responses truncated by max tokens > 1% → raise the cap or lower effort
- Hard-task accuracy flat across effort levels → effort is not the bottleneck; fix retrieval or tools
- Incomplete / max-token-truncated responses > 1%
- Reasoning tokens per request p95 by route — alert on > 50% week-over-week growth
- Visible-answer TTFT p95 vs SLO on thinking routes
When it goes wrong
7% of answers from a newly adopted reasoning model came back empty or cut off mid-sentence, all on the hardest tickets.
Blank-response counter and the API's incomplete status / 'max_output_tokens' reason in logs.
max output tokens was copied from the old route (2,000); reasoning tokens consumed the whole allowance before any visible text.
Raised the cap to 25,000, set effort to medium, alerted on incomplete responses, and showed a 'still thinking' state in the UI instead of an empty reply.
Related terms
Where it shows up
Sources
- DeepSeek-R1: Incentivizing Reasoning Capability in LLMs via Reinforcement Learning — DeepSeek-AI, 2025paper
- DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models (introduces GRPO) — Shao et al., 2024paper
- Tulu 3: Pushing Frontiers in Open Language Model Post-Training (RLVR) — Lambert et al., 2024paper
- Scaling LLM Test-Time Compute Optimally can be More Effective than Scaling Model Parameters — Snell et al., 2024paper
- Reasoning models guide — OpenAI, 2026docs
- Extended thinking (budget_tokens, migration to adaptive thinking + effort) — Anthropic, 2026docs
Check yourself
Three questions. Every option is explained once you answer.