Prompt chaining
A fixed sequence of LLM calls, each feeding the next, with programmatic gates between steps — the first workflow pattern in Anthropic's 2024 list.
Prompt chaining splits a big task into small steps, each its own call to an Large Language Model (LLM), with each step's output feeding the next. Step 1 pulls facts from a document, step 2 writes a draft from those facts, step 3 checks the draft.
A programmer writes the steps in advance, so it's a workflow rather than an AI agent (see Workflow vs agent). Small focused steps are easier for the model to get right and easier for you to test one at a time.
An assembly line with an inspector between stations: each station does one job, and a bad part is pulled before it reaches the next.
Passing free-form text between links with no validation. One malformed output silently corrupts every later step; type the handoffs and gate them.
Each link has a narrow Prompt & system prompt, its own model (small and fast for extraction, stronger for synthesis) and a typed output via Structured output / decoding, so the next link receives valid input. Gates between links — schema validation, a length or citation check, a classifier — stop errors early instead of letting them compound. Anthropic's "Building effective agents" lists this as the first workflow pattern: trade latency for accuracy by making each call easier.
Errors multiply: four links at 97% each give 0.97⁴ ≈ 88.5% end to end if failures are independent, which is why gates and per-link evals matter. Latency is the sum of sequential links unless independent links run in parallel. Because each link is evaluable alone, a regression can be pinned to one link in an Eval harness. It is the default skeleton for tasks with a known decomposition and the frame into which an agentic step is inserted when one branch needs it.
3–5 links; JSON Schema output on every link that feeds code; deterministic gates (schema, citation present, length) before any LLM-based gate; cheapest model that passes each link's golden set; failed gate → one retry with the error message, then route to a fallback or human queue.
Developer-docs release notes: (1) small model extracts changed APIs from 30 merged PRs as JSON (1.1 s); (2) gate: every item must cite a PR number, else drop and log; (3) larger model drafts notes from the JSON (2.4 s); (4) gate: banned-claims and broken-link check. End to end 3.9 s, ≈ $0.02 per release. When notes went wrong, logs showed link 1 had mislabelled a deprecation — fixed in that prompt alone.
Offer it as the simplest design that works whenever the decomposition is known, and name where the gates go. Recommend per-link golden sets so regressions are local.
Follow-up to expect: "latency is too high" — parallelise independent links, shrink models on easy links, or merge two links whose gate never fires.
Numbers worth knowing
- End-to-end accuracy
- Π link accuraciesindependent errors: 0.97⁴ ≈ 88.5%, 0.99⁴ ≈ 96%
- Latency
- Σ sequential linksparallel links count once (the slowest)
How to decide
| When | Do this |
|---|---|
| The task splits into steps that are the same for every input | Chain them with a gate after each step. |
| Two links are independent | Run them in parallel (Anthropic's sectioning pattern). |
| A gate fails often on one link | Fix that link's prompt or model; don't add retries downstream. |
Cost maths
Cost = Σ links (in_i × p_in,i + out_i × p_out,i). Release-notes example (illustrative, Oct 2026): link 1 on a $1 / $5 model with 9k in / 600 out = $0.012; link 3 on a $3 / $15 model with 1.5k in / 500 out = $0.012; gates are code (≈ $0). ≈ $0.024 per release. A single big prompt on the $3 model costs ≈ $0.035 and fails more often because one call does both jobs.
Operating it
- A gate fails on >5% of inputs
- End-to-end latency exceeds the SLO while links sit idle (parallelise)
- One link needs input-dependent branching (add routing or an agentic step)
- Gate failure rate per link (alert > 5%)
- Per-link golden-set pass rate (alert on −3 points)
- End-to-end p95 latency (alert above SLO)
When it goes wrong
Product descriptions in an e-commerce catalog began citing dimensions from the wrong SKU.
Merchant reports, confirmed by a sample audit showing 3% of descriptions mismatched their spec sheets.
Link 1 returned free text; when the spec sheet had two tables, link 2 parsed the second one. No gate checked that the extracted SKU matched the input SKU.
Link 1 emits JSON with sku and source_row; a gate rejects any mismatch; mismatches dropped to 0 and 0.4% of items route to manual review.
Related terms
Where it shows up
Sources
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog
- Chain of Thought Prompting Elicits Reasoning in Large Language Models — Wei et al., 2022paper
Check yourself
Three questions. Every option is explained once you answer.