What actually makes a system an agent, rather than an LLM call with extra steps?
- 2 min — definition: control-flow ownership, loop, stop condition
- 3 min — Anthropic taxonomy: augmented LLM → five workflow patterns → agents
- 3 min — compounding reliability and token cost numbers
- 2 min — example with the exception path routed into a bounded loop
- 3 min — org scope: runtime ownership, build vs buy, autonomy review
Use the taxonomy from Anthropic's Building effective agents (2024). The building block is the augmented LLM: one model call with retrieval, tools and memory. Workflows orchestrate those calls along code-defined paths — prompt chaining, routing, parallelization (sectioning and voting), orchestrator-workers, evaluator-optimizer. An agent is a model that directs its own process: it reads state, picks a tool or answers, observes the result, and loops until a stop condition fires (the ReAct pattern, Yao et al. 2022). The test is one question: who owns the control flow, and is the step count knowable at design time? If your code decides the sequence, it is a workflow even with ten tools.
The reason the distinction matters is arithmetic. Per-step reliability compounds: 95% per step over 10 steps is 0.95¹⁰ = 60% end to end; 99% per step gives 90%. Anthropic reported agents use about 4× the tokens of a chat turn. Autonomy is paid for in reliability and cost before it pays back in coverage.
Insurance first-notice-of-loss intake. Extracting policy number, loss date and peril from a claim form is a fixed 3-call chain with a schema at each step — a workflow, p95 under 4 s, ~$0.01 per claim. The agent earns its place on the exception path: “the claimant cites an endorsement that isn't in the policy file”. Now the steps are unknowable in advance — search the document store, fetch the endorsement history, compare effective dates, request missing evidence — and real traces range from 3 to 14 tool calls. Route only that ~8% of claims into a bounded loop with a 15-step cap.
Frame autonomy as a dial with five notches — single call → chain → router → bounded loop with ≤10 tools → open-ended agent — and require evidence to move up a notch: an error analysis showing a class of real inputs the lower notch cannot handle, measured on the same golden set for success, p95 latency and cost per successful task.
Org scope. A platform team owns the loop runtime: step and token budgets, tool registry, tracing, eval gates. Product teams own their tools, prompts and eval sets. Build vs buy: buy the model and use an SDK (Claude Agent SDK, OpenAI Agents SDK, LangGraph) for the first product; once three teams ship agents, own the ~300-line loop yourself, because budgets, tracing and kill switches are the product, not the framework. Roll out with an autonomy review: any team moving a feature from workflow to agent presents the error analysis and the cost delta. Failure mode to name: teams call any tool-calling feature an agent and evaluate the final message only, while the failures live in the trajectory.
What the answer sounds like at each level
An agent runs a loop where the model picks tools until it decides it is done; a single call that returns one tool call is not an agent.
Distinguishes workflows (code-owned control flow) from agents (model-owned), uses the compounding-reliability maths to justify keeping most paths as workflows, and names stop conditions.
Treats autonomy as a dial gated by error analysis and cost per successful task, assigns ownership of the loop runtime vs tools vs evals across teams, and makes the build-vs-buy call on the agent framework explicit.
Follow-ups the interviewer will push
1Your product manager wants “agentic” on the roadmap. How do you decide which features get a loop?
Run an error analysis on 200 real inputs through the current workflow. Only error classes that need a variable number of steps — search until found, compare until consistent — justify a loop. Everything else stays a chain, and I quote the cost delta (≈4× tokens) up front.
2Now 10× traffic. What breaks first in the agent version?
Tail cost and tail latency, not the median: runs that hit the step cap dominate spend. I would cap steps from the observed p99 of successful runs, add a loop detector on repeated identical calls, and route easy requests around the loop with a classifier.
3Is an LLM that emits one function call per request an agent?
No — it is structured output with a side effect. The loop and the model's ownership of the next step are what make it an agent, and what change how you test it.
Red flags interviewers listen for
- Defines an agent as “an LLM with tools” without mentioning who owns the control flow.
- Cannot say why a 10-step agent at 95% per-step reliability fails 40% of the time.
- Recommends a multi-tool agent for a process whose flowchart fits on one slide.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Precision of definition | Separates workflow and agent by control-flow ownership, not by tool count. |
| Quantitative reasoning | Computes end-to-end reliability from per-step reliability and quotes a token-cost multiple. |
| Judgement | Defaults to the lowest autonomy notch and states the evidence needed to move up. |
| Org scope | Names who owns the loop runtime, tools and evals, and when to stop using a framework. |
Sources
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog
- ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022paper
- How we built our multi-agent research system — Anthropic, 2025blog