When should you NOT build an agent — and use a workflow instead?
- 2 min — rule: if the flowchart is drawable, it is a workflow
- 4 min — the five workflow patterns with one example each
- 3 min — variance cost of a needless loop, in tokens and p95
- 2 min — exception-slice design: workflow core, bounded agent branch
- 2 min — org default and review gate
Default to a workflow whenever you can draw the flowchart in advance. Anthropic's Building effective agents (2024) lists five workflow patterns that cover most production needs: prompt chaining (fixed sequence with programmatic gates), routing (classify, then send to a specialised path), parallelization (sectioning independent subtasks, or voting — run the same task 3–5 times and aggregate), orchestrator-workers (a model decomposes, code dispatches), and evaluator-optimizer (one call generates, another critiques against criteria, loop until pass). Each has a bounded step count, so cost and p95 latency are predictable and failures are enumerable.
The cost of an unnecessary agent is variance. If a task needs 4 steps and the agent takes 4–11 (observed p50 5, p95 9), you pay ~2× the median tokens at the tail and the eval must now cover paths you never designed.
Financial filings: extracting segment revenue from 10-K filings. The shape never changes — locate the segment note, extract the table to a schema, validate that segments sum to consolidated revenue within 0.5%, flag mismatches. Built as a 4-call chain with a programmatic validation gate it runs at p95 6 s and ~$0.04 per filing. The open-ended part — “segments don't reconcile; find out why (restatement? discontinued operation?)” — is ~6% of filings and goes to a bounded agent with search and diff tools and a 12-step cap.
Start with the simplest thing — one well-engineered call plus retrieval — and promote only on evidence: an error class the fixed path cannot handle, shown on the golden set with success, p95 and cost per successful task side by side. Evaluator-optimizer is the most under-used pattern: it gives you iterative refinement with a hard iteration cap (2–3 rounds) and a programmatic or rubric stop signal, which is most of what teams wanted from an agent.
Org scope. Make “workflow first” a platform default, not a per-team argument: the shared orchestration library ships the five patterns as primitives, and a design review requires the error analysis before a loop is approved. Build vs buy: workflows need no agent framework — plain code or a durable workflow engine you already run. Failure mode: an agent wrapped around a deterministic process; the same input takes 4 steps one run and 9 the next and no one can explain why.
| Pattern | Who owns flow | Steps | Use when |
|---|---|---|---|
| Prompt chaining | Code | Fixed N | Known sequence |
| Routing | Code + classifier | 1 + branch | Distinct input types |
| Parallelization | Code | N concurrent | Independent parts / voting |
| Orchestrator-workers | Model plans, code runs | Varies | Subtasks unknown up front |
| Evaluator-optimizer | Code loop | ≤3 rounds | Clear quality criteria |
| Agent | Model | Unbounded (capped) | Path truly unknowable |
What the answer sounds like at each level
If the steps are known in advance, write it as a chain of LLM calls instead of an agent; it is cheaper and easier to test.
Names the five workflow patterns (chaining, routing, parallelization, orchestrator-workers, evaluator-optimizer), picks one for the example, and quantifies the variance cost of an unnecessary loop.
Makes workflow-first a platform default with shared primitives and a design-review gate, promotes to agents only on error-analysis evidence, and reserves the loop for a measured exception slice.
Follow-ups the interviewer will push
1Evaluator-optimizer looks like an agent loop. Why is it a workflow?
The code owns the loop: fixed generator and critic roles, a hard cap of 2–3 iterations, and a programmatic or rubric pass condition. The model never chooses which tool runs next.
2Now the provider has a 30-minute outage. Which design degrades better?
The workflow: each step is a known call I can route to a fallback model validated on that step's eval. An agent's behaviour on a different model is a new system and needs its own trajectory eval before failover.
3When would voting (parallelization) beat a single better prompt?
When errors are uncorrelated across samples — e.g. a classification with 85% single-run accuracy and ~independent errors goes to ~94% with majority-of-3. If errors are correlated (same misleading passage), voting buys nothing; check agreement rate first.
Red flags interviewers listen for
- Says “it depends” without naming the observable that decides (known step count, error classes).
- Cannot name more than one workflow pattern besides chaining.
- Proposes an agent and has no plan to compare it against a workflow baseline on the same eval set.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Pattern fluency | Names and correctly applies routing, parallelization and evaluator-optimizer, not just chaining. |
| Evidence | Promotes to an agent only on error analysis with success, p95 and cost per successful task. |
| Design | Isolates the open-ended slice behind a classifier with its own step cap. |
| Org scope | Makes the default a platform primitive and a review gate, not a team preference. |
Replacing an agent with a workflow
Illustrative exampleOur internal developer-docs assistant generated weekly changelog digests for 40 engineering teams. It had shipped as an agent with 8 tools. Across 1,200 runs over six weeks, steps per run ranged from 4 to 25, cost per digest varied 6× (p50 $0.18, p95 $1.10), and 9.5% of runs ended on the 25-step cap after looping on the same search tool.
I was the tech lead on a team of four. The goal: make digests predictable without losing quality, and decide whether agency was earning its keep. The original author, a senior engineer, disagreed strongly — he argued the agent “handled weird weeks” and a chain would be brittle.
I pulled 300 traces and did the error analysis myself: 91% of successful runs followed the same four steps in the same order; nearly all variation was failure. Rather than argue, I asked the dissenting engineer to pick the 30 “weird weeks” he worried about, and we put them in the golden set alongside 120 normal weeks. Two engineers rebuilt the core as a chain — fetch merged PRs, fetch release notes, draft, evaluator-optimizer check against a 5-point rubric with at most 2 revisions. The dissenter built the escape hatch: a bounded agent (8-step cap) for weeks a classifier flagged as anomalous. I wrote the comparison harness and owned the go/no-go.
On the 150-task golden set (n=150, 3 trials each), rubric pass rate went from 82% to 85%, cost per digest p95 fell from $1.10 to $0.21, and step-cap exits went from 9.5% to 0. The anomaly branch triggered on 7% of weeks. The team adopted the rule that new features start as workflows. What I'd do differently: I spent two weeks arguing from intuition before running the error analysis; I'd run it on day one, and I'd have let the dissenter own the golden-set selection from the start — his 30 hard cases were the most valuable part of the eval.
Sources
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog