Workflow vs agent
Workflows run LLMs through code paths you wrote; agents let the model pick the path. Anthropic's 2024 split; trade predictability for flexibility.
There are two ways to build with an Large Language Model (LLM). In a workflow, a programmer writes the steps (classify the claim, extract the fields, check them) and the model fills in each one. In an agent, the model gets a goal and tools and decides the steps itself.
The split comes from Anthropic's "Building effective agents" (December 2024). Workflows are predictable, cheap and easy to test. Agents handle open-ended tasks where nobody can list the steps in advance, but they cost more, take longer and fail in stranger ways.
A recipe versus a chef. The recipe gives the same dish every time; the chef improvises with what's in the fridge, but you can't predict the bill or the time.
Treating "agent" as the advanced, therefore better, choice. Many agent projects get cheaper, faster and more accurate when collapsed back into a router plus two chains.
Anthropic's taxonomy (Dec 2024) names five workflow patterns before autonomous agents. Prompt chaining: fixed sequence with programmatic gates (Prompt chaining). Routing: a classifier (Intent classification & routing) sends each input to a specialised prompt or a cheaper model. Parallelization: sectioning (independent sub-tasks run at once) or voting (the same task N times, aggregated). Orchestrator-workers: a model decides the sub-tasks at run time and synthesises the results (Orchestrator–worker pattern). Evaluator-optimizer: one call generates, another judges against criteria, loop until pass (Reflection & self-critique). Autonomous agents run a tool loop with environment feedback and stop conditions.
The axis is who owns control flow. Code-owned paths give bounded latency, near-constant cost and per-step evals. Model-owned paths cover the long tail but turn cost and latency into distributions and widen the failure surface (loops, wrong tool, Prompt injection (indirect) via tool output). The production default is hybrid: a deterministic skeleton with one agentic step where the path truly can't be enumerated, wrapped in Stop conditions & step budgets and Trajectory evaluation.
Start with a router plus 2–3 chains, each link with typed output and a gate. Add an agentic step only for the branch whose path depends on tool results, capped at ~15 turns and $0.50. Keep one Golden set per branch so you can see which branch regressed.
Insurance claims intake. First notice of loss → extract 14 fields → validate policy number and dates → route by claim type. Steps never change, so it is a 3-link chain plus a router: p50 3.1 s, p95 4.0 s, ≈ $0.006 per claim. Coverage disputes — where the next check (policy wording, prior claims, adjuster notes) depends on what the last lookup found — go to an agent: p50 6 tool calls, p95 19, ≈ $0.04–$0.30 per case. The agent handles 8% of volume and 60% of the model spend.
When asked "should this be an agent?", draw the flowchart first. If you can draw it, recommend a workflow and name the pattern (chain, router, parallel, evaluator loop). Hand the model the pen only for the box you cannot draw, and state the caps you'd put on it.
The follow-up is usually "what if the steps change monthly?" Answer: that's a config or routing problem, not a reason for autonomy; autonomy is for paths that change per input.
| Pattern | Path chosen by | Use when | Cost shape |
|---|---|---|---|
| Prompt chaining | Code | Fixed decomposition | N calls, constant |
| Routing | Code + classifier | Distinct input types | 1 + one branch |
| Parallelization | Code | Independent parts / votes | N calls, parallel |
| Orchestrator-workers | Model (sub-tasks) | Parts unknown upfront | Grows with fan-out |
| Evaluator-optimizer | Code loop, model judge | Clear pass criteria | 2 × rounds |
| Autonomous agent | Model | Open-ended path | A distribution |
Workflow
- Code decides the steps
- Each step testable alone
- Latency and $ bounded
- Breaks on unforeseen inputs
Agent
- Model decides the steps
- Path varies per input
- Latency and $ are a distribution
- Covers the long tail
Numbers worth knowing
- Workflow latency
- Σ of link latencies, ±15%fixed 3–5 link chain, no retries
- Agent latency spread
- p95 ≈ 3–5× p50typical tool-using agent with caps; measure your own
- Accuracy compounding in a chain
- 0.97⁴ ≈ 88.5%4 links each 97% correct, errors independent
How to decide
| When | Do this |
|---|---|
| You can draw the flowchart and it is the same for every input | Workflow: chain, router or parallel fan-out. |
| Inputs fall into a few distinct types with different handling | Routing to specialised prompts or smaller models. |
| Quality improves measurably with feedback against clear criteria | Evaluator-optimizer loop with a 2–3 round cap. |
| The next step depends on what the last tool returned | Agentic step inside the workflow, with Stop conditions & step budgets. |
Cost maths
Workflow cost = Σ links (in × p_in + out × p_out), roughly constant. Agent cost = E[turns] × E[cost per turn], and cost per turn grows with history. Claims example at $3 / $15 per M (illustrative, Oct 2026): chain of 3 links, 1.2k in + 150 out each → 3 × (0.0036 + 0.00225) ≈ $0.018 per claim, ≈ $0.006 with a small model on two links. Agent branch: median 6 turns ≈ $0.04, p95 19 turns ≈ $0.30. At 100k claims/month with 8% agentic: workflow ≈ $600, agent ≈ $1,000 (mean ≈ $0.12) — the 8% costs more than the 92%.
Operating it
- One workflow branch accumulates >10 hand-written special cases
- Router confidence below threshold on >10% of inputs
- Agent branch share of spend exceeds 3× its share of volume
- Share of traffic and of spend per branch (alert if agent spend share > 3× its volume share)
- Per-link pass rate on the golden set (alert on −3 points)
- Agent-branch p95 latency (alert above the SLO, e.g. 30 s)
When it goes wrong
A contract-review assistant built as a free-form agent took 40–90 s per NDA and occasionally skipped the governing-law check.
Trajectory sampling showed the same 5 steps in 97% of runs, in varying order, with check_governing_law missing in 6% of traces.
A fixed procedure had been implemented as an agent, so the model re-derived the plan every run and sometimes dropped a step.
Rebuilt as a 5-link chain with a schema gate after each link; p50 fell to 9 s, the missing-check rate to 0, and the agent was kept only for non-standard clauses.
Related terms
Where it shows up
Sources
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog
- τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains — Yao et al., 2024paper
Check yourself
Three questions. Every option is explained once you answer.