One line per term, the default to reach for, and the number to remember. Print it or save it as a PDF.
A system where the LLM directs its own steps and tool calls in a loop until the goal is met or a stop condition fires (Anthropic, Dec 2024).
Default: One model in a single The agent loop (ReAct) via the provider's native tool-use API (Anthropic Messages, OpenAI Responses) or a thin SDK (Claude Agent SDK, OpenAI Agents SDK, LangGraph).
Tokens per agent interaction vs a chat turn: ≈4× (Anthropic internal data, June 2025)
Workflows run LLMs through code paths you wrote; agents let the model pick the path. Anthropic's 2024 split; trade predictability for flexibility.
Default: Start with a router plus 2–3 chains, each link with typed output and a gate.
Workflow latency: Σ of link latencies, ±15% (fixed 3–5 link chain, no retries)
The model emits a structured call (tool name + JSON arguments); your application validates and runs it, then returns the result to the model.
Default: Native tool-use API with strict/structured arguments where offered; 3–10 tools; JSON Schema with enums and required fields; identity and tenant injected server-side; 10 s default tool timeout; idempotency key on every write; tool results capped at a few thousand tokens with a truncation hint; parallel tool calls on for independent reads.
Tokens per tool definition: 100–500 (name + description + schema; measure with the provider's token counter)
The core cycle of every agent: model decides, host runs a tool, result is appended, repeat until a final answer or a stop condition.
Default: Host-enforced caps: 25 turns, 300k cumulative tokens, 10 min, $1.
Input at turn t: prefix + (t−1) × Δ (history kept verbatim; Δ = tokens added per turn)
Having the agent write out the steps for a goal before acting, then executing and re-planning when a tool result contradicts the plan.
Default: For tasks over ~10 steps: one planning call that writes a numbered todo list into state; execute steps with the same or a cheaper model; re-plan on any failed check or empty result; show the plan for approval when the task writes to shared systems.
Planning-call overhead: 1 extra call, ~1–3k output tokens (plan of 5–15 steps)
The model critiques its output or trajectory and revises it; it helps when the critique uses an external signal (tests, validators), not self-review alone.
Default: Evaluator-optimizer with a deterministic checker first (tests, JSON Schema, regex policy), then an LLM-as-judge with a written rubric only for what code can't check.
Self-Refine average gain: ~20% absolute (7 tasks, GPT-3.5/4, Madaan et al. 2023)
A fixed sequence of LLM calls, each feeding the next, with programmatic gates between steps — the first workflow pattern in Anthropic's 2024 list.
Default: 3–5 links; JSON Schema output on every link that feeds code; deterministic gates (schema, citation present, length) before any LLM-based gate; cheapest model that passes each link's golden set; failed gate → one retry with the error message, then route to a fallback or human queue.
End-to-end accuracy: Π link accuracies (independent errors: 0.97⁴ ≈ 88.5%, 0.99⁴ ≈ 96%)
Several LLM agents with separate contexts, instructions and tools cooperating on one task: strong for parallel breadth, ~15× chat tokens.
Default: Single agent first.
Quality gain on breadth-first research: +90.2% (multi-agent vs single Opus 4, Anthropic internal eval, June 2025)
A lead model splits a task into sub-tasks at run time, dispatches them to workers (calls or agents), then merges results — Anthropic's 4th pattern.
Default: Brief template with objective, inputs, tools, output schema, boundaries and effort budget; workers return ≤500-token structured results; fan-out cap 10, recursion depth 1; workers read-only unless each owns a disjoint write scope; synthesis step checks for conflicting facts.
Effort scaling used by Anthropic's research agent: 1 / 2–4 / 10+ sub-agents (fact-finding / comparison / complex research, June 2025)
One agent transfers control of a conversation or task to another agent or a human, passing a structured briefing and swapping tools and permissions.
Default: Handoff as a tool with a typed briefing (intent, entities, what was tried, customer's words); receiver gets its own instructions and tools only; max 2 agent-to-agent handoffs per conversation, then a human; human handoffs create a ticket with the briefing and a link to the trace.
Briefing size: 150–400 tokens (structured summary vs 2–8k for a raw transcript)
Model Context Protocol: an open JSON-RPC standard connecting AI apps (hosts) to servers exposing tools, resources and prompts. Linux Foundation since Dec 2025.
Default: Pin the protocol revision your host supports and test against it.
Active public MCP servers: >10,000 (Anthropic, Dec 2025)
How an agent keeps information beyond one context window: working memory in context, plus long-term stores it writes to and retrieves from.
Default: Per-tenant, per-user facts table (text, source, created_at, ttl, confidence) loaded top-10 by relevance × recency at session start; explicit remember/forget tools; writes from tool output require provenance; no instructions stored as memory; deletion API wired to account erasure.
Memory loaded per session: 300–1,000 tokens (top 5–15 facts; vs 10k+ for raw past transcripts)
Deciding exactly what enters the model's context at each step — instructions, tools, retrieved data, history — to maximise signal per token.
Default: Stable prefix (instructions → tools) first and cached; tool outputs capped at ~2–5k tokens with a truncation hint; retrieval via tools rather than preloading; compaction at ~70% of the window preserving decisions, open questions and IDs; per-step token logging with task success.
Models showing length-dependent degradation: 18 of 18 tested (Chroma "Context Rot", 2025)
An agent operating a graphical interface from screenshots via mouse and keyboard actions — the fallback for software with no API.
Default: Disposable VM or container browser per task; 1280×800 or provider-recommended resolution; domain allowlist; credentials injected by a broker, never typed from memory; confirmation required for submit/purchase/send/delete; screenshot trail stored per step; step cap ~50 per task.
Tokens per screenshot: ≈ w × h / 750 (Claude vision; 1280×800 ≈ 1,365 tokens)
An isolated environment where an agent's code and actions run with scoped files, network and credentials, so mistakes can't reach real systems.
Default: Ephemeral microVM (Firecracker) or gVisor container per run; workspace-only mount; egress denied except an allowlisted package proxy; metadata endpoints blocked; per-run tokens with 15-minute expiry; 2 vCPU / 4 GB / 10-minute limits; logs of every blocked syscall and connection shipped off-box.
Firecracker microVM boot: ≈125 ms (NSDI 2020 paper)
Checks around an LLM or agent that block, modify or escalate unsafe inputs, outputs and actions — layered, with the critical ones in code.
Default: Deterministic action limits in every write tool; schema + grounding check on outputs that state policy, price or coverage; a small injection classifier on retrieved and tool content; PII filter on output; $ and turn caps; every block logged with layer and reason; monthly precision/recall review on 200 labelled samples.
Latency: rule check: < 1 ms (regex, schema, allowlist)
A person reviews, approves or corrects an AI's work at chosen points — usually before irreversible or high-stakes actions.
Default: Risk rules in code (amount, irreversibility, external recipient, changed bank details) select the gated calls; state checkpointed before the pause; approval UI shows action, arguments, evidence and the agent's reason; approve/edit/reject with a required reason on reject; SLA timer that escalates; every decision stored as a label.
Healthy gated share: 2–10% of actions (risk-based gating; >30% means the rule is too broad)
Rules that end an agent loop: task done, or a cap on turns, tokens, time, spend, repeated actions or errors — each with a logged reason.
Default: Turn cap ≈ 1.3× p99 of successful runs; token and $ caps likewise; wall-clock cap; repeat detector (3 identical calls); error budget (3–4 consecutive failures); explicit finish tool with a result schema; stop_reason enum logged on every run; graceful partial-result template.
OpenAI Agents SDK default: max_turns = 10 (as of Oct 2026)
Running agent workflows so progress is checkpointed and they resume after crashes, deploys or long waits without redoing completed side effects.
Default: Short runs (< 5 min, no human wait): checkpoint state to Postgres after each turn.
Temporal history warning: 10,240 events or 10 MB (per workflow execution, Temporal docs)
Evaluating an agent's whole path — tool choices, arguments, order, safety, efficiency — and its end state, over repeated trials (pass^k).
Default: 100–300 tasks from real traces; tools replayed from fixtures or run in a sandbox; end-state check per task; trace assertions for required, forbidden and ordered steps; 4 trials per task reporting pass^1 and pass^4; cost and steps per success; run on every prompt, model or tool change as a CI gate.
pass^k with independent trials: p^k (p = 0.8 → pass^8 ≈ 0.17)
Designing an agent's tools — names, descriptions, parameters, outputs, errors — so the model picks and uses them correctly and cheaply.
Default: 5–15 task-shaped tools per agent, namespaced by system; descriptions that say when to use and when not to; enums for closed sets; identity from the session, not arguments; outputs ≤ 2–5k tokens with pagination and a truncation hint; response_format concise|detailed; errors with a fix suggestion; reads and writes split; writes idempotent; tool evals in CI.
Tool response cap in Claude Code: 25,000 tokens default (Anthropic, Sept 2025)
What an agent run costs in tokens, money and time. It grows quadratically with turns unless cached, so manage it as a distribution per success.
Default: Prompt caching on the stable prefix and a moving breakpoint on history; tool outputs trimmed to ≤2–5k tokens; compaction at 70% of the window; per-run caps ($1–2, 300k tokens); per-tenant monthly quotas in the gateway; cheaper model for classification and extraction steps; batch API for offline agents; dashboards of cost per successful task p50/p95 by agent version.
Cumulative input over T turns: T·P + Δ·T(T−1)/2 (P = prefix, Δ = tokens added per turn, no compaction)
Serving many customers from shared infrastructure while keeping their data, quotas and performance isolated. Isolation tier is a cost vs risk choice.
Default: Pooled Postgres with RLS (FORCE ROW LEVEL SECURITY, SET LOCAL app.tenant_id per transaction); vector store with tenant partitioning (Qdrant is_tenant payload index, Pinecone namespaces, or a tenant_id filter in pgvector with RLS); tenant id in every cache key, log, trace and metric; per-tenant rate limits; silo tier for regulated customers; cross-tenant test suite in CI.
Pooled vs silo infra cost: ~10–100× cheaper per small tenant (a dedicated cluster has a fixed floor regardless of data size)
One tenant's heavy load (a bulk import, a runaway agent) degrading latency or quota for everyone sharing the same infrastructure.
Default: Per-tenant token buckets (tokens/min and requests/min) at the LLM gateway; separate queues and worker pools for batch/ingestion vs interactive; weighted fair queuing across tenants; per-run token and step caps for agents; shuffle sharding for stateful workers; per-tenant metrics on every request.
Shuffle-shard overlap, 8 workers, shard size 2: 1 in 28 (chance another given tenant shares both of your workers)
Access control list: who may see a document. In RAG, ACLs must be enforced at retrieval time or the model will quote what the user can't open.
Default: Principals copied onto every chunk; nested groups flattened at ingest; user groups from the IdP (Okta, Entra ID) cached ≤5 min; filter in the ANN query (Qdrant/OpenSearch filters, pgvector + RLS); webhook-driven ACL sync plus 15-min ACL reconcile; source re-check for restricted spaces; ACL hash in cache keys; deny if group resolution fails.
Revocation propagation target: ≤5 minutes (webhook sync; nightly-only sync means up to 24 h exposure)
When a security check can't run (auth service down, classifier timeout), deny by default. The opposite, fail-open, keeps serving and skips the check.
Default: Every security check returns allow/deny/error with error → deny in code; check services have their own SLO and in-region replicas; positive decisions cached ≤60 s; denial UX with retry; fail-open exceptions require a named owner and are listed in the threat model; chaos test that kills each check service and asserts denial.
Positive-decision cache: ≤60 s (trades a short revocation lag for availability)
Text that the model reads as instructions: typed by a user (direct) or hidden in a page, email or doc the agent reads (indirect). No filter fully fixes it.
Default: Trifecta review per agent at design time; tools scoped per task (read-only by default, single repo/folder); egress allowlist for HTTP and mail tools; markdown images and links rendered only for allowlisted domains; human approval for send/pay/delete/share; plan-then-execute or dual-LLM for agents processing untrusted content; injection classifier (e.g.
Residual success rate of filtered attacks: non-zero under adaptive attack (published evaluations repeatedly bypass classifier-only defences)
Getting data out of a system without authorisation. For agents: via a tool call, a fetched URL, or a rendered markdown image carrying data in its query string.
Default: Network egress allowlist for agent sandboxes and HTTP tools (deny by default); chat UI renders images only from your CDN or via an image proxy (CSP img-src restricted); links to non-allowlisted domains shown inert with the domain visible; approvals for any write outside the tenant; outbound request log keyed by agent run id; alerts on long query strings to new domains.
Channels to review per agent: 4 classes (tool requests, external writes, rendered output, sandbox network)
Give each user, service and agent only the permissions its current task needs, for only as long as it needs them. Limits damage when it's tricked or buggy.
Default: Per-task credentials from a broker (AWS STS, GCP short-lived tokens, Vault dynamic secrets); OAuth scopes per MCP server per the MCP authorization spec; read-only by default; separate dev/prod accounts with no cross-access; approval gates for send/pay/delete/share; quarterly review of agent permissions vs actual usage.
Credential lifetime: 15 min–1 h (AWS STS sessions start at 15 min; shorter limits leaked-token value)
How much breaks when one thing fails or is compromised: how many users, tenants, records or actions. Design to keep it small and known.
Default: Cell-based deployment for multi-tenant serving; staged rollout for code, config, prompts and flags; per-tenant and per-task credentials; agent caps (tokens, steps, spend, rows per tool call); blast-radius statement in every design doc; detection SLOs on canary gates.
Tenants per cell: a few hundred (small enough that one cell's outage is a contained incident)
Stacking independent controls so one failing (and each will sometimes) doesn't mean compromise. For LLM agents, no single layer is reliable alone.
Default: Layer inventory per agent mapped to OWASP LLM Top 10; deterministic limits inside every side-effecting tool (amount caps, recipient allowlists, row caps); least-privilege credentials; egress allowlist; approvals for irreversible actions; classifier layer for detection; red-team suite (≥200 attacks) run per release with per-layer catch rates.
Red-team suite size: ≥200 attacks (covering direct, indirect, multi-turn and tool-output injection)