Workflows before agents: pick the simplest pattern that works

Picture an insurance team that prototypes a claims-intake assistant as a tool-calling AI agent. In the demo it reads a customer email, looks up the policy, checks coverage, drafts a reply. Suppose the logs from the first week of internal use show a median run of 9 model calls and 14 seconds, the p95 takes 31 calls and 70 seconds, and 80% of runs follow the same four steps in the same order. Those four steps are a workflow. The team is paying agent prices — in money, latency and debuggability — for a fixed pipeline.

The cure is not new. Anthropic’s December 2024 essay Building effective agents (Schluntz and Zhang) laid out a taxonomy that most production teams now use, whether they cite it or not: an augmented LLM as the building block, five workflow patterns, and agents at the top. Its core advice was to start with the simplest thing and add complexity only when it demonstrably improves outcomes. This post follows that taxonomy rung by rung, adds the numbers that make the trade-offs concrete, and keeps both patterns that shorter summaries usually drop: parallelization and evaluator-optimizer.

The distinction underneath is the workflow versus agent split. In a workflow, your code decides the sequence and the model fills in each step. In an agent, the model decides the sequence at runtime. Everything below is a consequence of who owns the control flow.

Before the patterns, the price list, because the spread is the argument. Assumptions, all stated: one “standard” call is 1,500 input and 400 output tokens; a large model at $3 / M input and $15 / M output (illustrative list prices, Oct 2026), so one call costs about $0.011 and takes about 2 s; a small classifier model at $1 / $5 per M; no prompt caching unless noted.

The agent row is where intuition fails. Each step re-sends the whole conversation, so input tokens grow with the square of the step count: step k reads roughly 1,500 + 1,000·k tokens. Ten steps read about 70k input tokens ($0.21) plus 4k output ($0.06), so $0.27. Forty steps read 880k input tokens ($2.64) plus 16k output ($0.24), so $2.88. Same task, same model, 10× cost spread. Caching the stable prefix at 0.1× read price cuts the forty-step run to roughly $0.65 (reads at 0.1×, each step’s new 1k tokens written at 1.25×), which is why agent cost work starts with cache hit rate.

Anthropic’s building block is a single model call augmented with retrieval, tools and memory. Classification, extraction, summarising one document, converting free text into a JSON record with structured output: one call. It has one latency number, one cost number and one place to look when it is wrong. A golden set for it takes an afternoon.

Climb when: improving one behaviour in the prompt keeps breaking another, or the task has stages whose quality you need to check separately.

Prompt chaining splits the task into fixed steps with code between them: extract the claim facts, validate them against the policy schema in code, then draft the reply from the validated facts only. The code between steps is the quiet superpower — it can reject an empty extraction, call a deterministic coverage function, or stop early. Each step gets a narrower prompt, so the easy steps can run on a model at a third of the price.

Cost of climbing: latency adds serially (3 steps ≈ 3× the single call). Climb when: inputs need materially different handling and the chain is growing if branches around prompts.

Routing puts intent classification at the front and sends each input to a specialised path: new-claim intake, claim status, coverage questions, complaints. Each path stays simple and gets its own eval set. In production the classifier is a model call, not a keyword list, and three decisions matter more than the prompt wording.

• Cheap model first. Classification emits a few tokens, so a small model (Haiku-class, assume $1 / $5 per M) costs about a third of a large one per call and answers in 300–500 ms instead of 1–2 s. On a 4–8-way intent problem, small models typically reach within a few points of large ones; measure that on your own labelled sample of 200+ messages before you trust it.
• Give it an exit. Add an unsure label and tell the model to use it when two routes fit equally. Self-reported confidence scores are poorly calibrated; an explicit abstain label is easier to evaluate.
• Escalate the minority. Send only unsure and unparseable outputs to the large model. If 10% escalate, blended cost is about 0.9 × small + 0.1 × (small + large), and p50 latency stays at the small model’s.

Note the defaults: an invalid label becomes unsure, and a final unsure falls to the path a human watches (complaints), not to a generic answer. A misrouted request fails no matter how good the downstream prompt is, so track router accuracy and the escalation rate as first-class metrics; an escalation rate drifting from 10% to 25% is usually a new intent your routes do not cover.

Climb when: the work splits into independent pieces, or one answer is not reliable enough on its own.

The first pattern most ladders omit. Anthropic names two variants. Sectioning runs independent subtasks at once: check a claim email for coverage, fraud signals and missing documents in three concurrent calls, each with a focused prompt. Voting runs the same task several times and aggregates: five independent reads of whether a clause excludes flood damage, escalating to a human if fewer than four agree.

Parallelization is the only rung that adds cost without adding much latency: five concurrent calls cost 5× but finish in roughly the time of the slowest one, about 1.25× a single call. Use voting where a false negative is expensive and the task is short; it is wasted on long generative tasks where the five answers cannot be compared mechanically.

The second commonly omitted pattern: one call generates, another evaluates against explicit criteria and returns feedback, and the generator revises. It is reflection with the loop bounded by code. It pays off when you can state the criteria crisply and a human reviewer’s feedback would measurably improve the draft — customer letters that must cite the exact policy clause, or translations that must keep defined legal terms.

Cap it at 2–3 rounds and stop as soon as the evaluator passes; most of the gain comes in the first revision. Without a cap, an evaluator that is stricter than the generator can satisfy produces an expensive infinite polish loop, which is a stop-condition bug wearing a quality badge.

In orchestrator-workers, one call plans the subtasks and workers execute them, often in parallel, before a final synthesis. The difference from sectioning is that the subtasks are not known in advance: a multi-vehicle accident claim may need three policy lookups or eleven.

Your code still owns the boundary. The plan is data: cap the number of subtasks (8 is a sane default), validate each against an allow-list of tools, reject plans that request write actions, and set a budget for the whole run. The model chooses what to do inside a box you drew.

Climb when: the next step depends on what the last one discovered, repeatedly, so any plan made up front is wrong by step two.

A full agent runs an agent loop — reason, act with a tool, observe, repeat — the ReAct shape from Yao et al. (2022). It is the right shape for open-ended work against an environment that answers back: debugging a failing test suite, navigating an unfamiliar codebase, operating a computer interface.

It is also where every property you liked about lower rungs degrades at once. Cost becomes a distribution (the 10× spread above). The same input takes 4 steps or 40, so failures are hard to reproduce and evaluation moves from checking answers to checking trajectories. And the agent needs hard stop conditions: a step cap (25 is a reasonable start), a spend cap per run, a wall-clock limit, and a definition of done that code can check.

Two questions outweigh the rest. Reversibility: autonomy multiplies actions taken without a human looking, so if those actions send email, pay claims or delete records, every rung widens the blast radius. Evaluability: a chain’s wrong answer points to one step; an agent’s wrong answer points to a path, and you need 3–5× more eval examples to see stable numbers when outcomes vary run to run.

The healthy production shape is mixed: a cheap router in front, fixed chains for the 80–90% of traffic with known shape, and a bounded agent behind one route for the genuinely open-ended rest. The agent serves the minority that needs it; the predictable paths serve everyone else.

Climbing down is the move teams forget. Log every agent trajectory, cluster them by the sequence of tool names, and look at the top clusters. In the claims example, one cluster covered 80% of runs with the same four tools in the same order. Extracting it into a chain cut median latency from 14 s to about 5 s and median cost by roughly two-thirds, and left the agent only the 20% of claims that actually needed it. Agent logs are the best source of workflow designs you will ever get.