How do you design human-in-the-loop approvals for an agent?
- 2 min — mechanism and risk-based gating
- 2 min — legal context: Air Canada
- 3 min — procurement example and card design
- 3 min — metrics, relaxation with rule of three, reviewer capacity
- 2 min — compliance/ops/platform ownership
Human-in-the-loop inserts a person at defined decision points: the agent proposes, execution pauses, a human approves, edits or rejects, and the run resumes. Anthropic's 2024 guidance is to pause for human feedback at checkpoints and blockers, not everywhere. Gate on risk — irreversibility, money, external communication, access changes, regulated decisions — not on step count, or reviewers rubber-stamp.
Why it matters legally: in Moffatt v. Air Canada (2024) a tribunal held the airline responsible for what its website chatbot told a customer. What an agent commits to externally is what the company commits to.
Procurement agent for an operations team. It drafts purchase orders freely; it pauses for approval on any PO above $5,000, any new vendor, and any change to vendor bank details (a classic fraud vector — never auto-approved). The approver sees a one-screen card: proposed action, diff against current state, the 3 pieces of evidence the agent used with links, and its stated reason — not the 40-step transcript. Approve, edit inline, or reject with a reason; rejections feed back to the agent to re-plan. Unanswered requests escalate after 24 h.
Approvals require durable execution underneath — waits last hours or days. Measure approval rate, edit rate, rejection reasons and time-to-approve per gate. A gate with >98% unedited approvals over n ≥ 500 and zero severe errors is a candidate to auto-approve under a threshold (zero errors in 500 bounds the true error rate below ~0.6% at 95% confidence, by the rule of three). Edits are labelled data for the eval set and the data flywheel. Budget reviewer time explicitly: 2,000 gated actions/day × 40 s ≈ 22 hours ≈ 3 FTE.
Org scope. Risk/compliance defines which actions are gated and who may approve (segregation of duties: the requester can't approve their own agent's PO). Operations staffs the reviewers and owns the time-to-approve SLO. The platform team owns the approval service — durable waits, escalation, audit log — reused by every agent, with approval cards delivered where people already work (Slack/Teams, email for EU/US teams, WhatsApp where that is the primary channel). Build vs buy: build on the workflow engine's signals; don't buy a separate approvals product. Failure mode: approval fatigue, and cards that show too little for a real decision — the human becomes a liability shield, not a control.
Assumptions: 12k agent actions/day, 17% gated → ~2,000 approvals/day at 40 s each = 22.2 reviewer-hours ≈ 3 FTE at 7.5 productive hours. Auto-approving POs < $1,000 from known vendors (60% of gated volume) after n ≥ 500 clean approvals cuts it to ~1.2 FTE.
What the answer sounds like at each level
Pause the agent before risky actions and ask a person to approve.
Gates by risk class, designs a decision-ready approval card, measures approval/edit rates and relaxes gates on evidence with explicit statistical bounds.
Splits gate policy (compliance), reviewer staffing (ops) and approval infrastructure (platform), budgets reviewer FTEs, and enforces segregation of duties.
Follow-ups the interviewer will push
1Reviewers approve 99.5% in under 5 seconds. Problem?
Yes — that's rubber-stamping. Either the gate is too broad (auto-approve low-risk classes) or the card is wrong. Sample-audit approved actions and seed known-bad proposals to measure whether reviewers catch them.
2Now 10× volume. Reviewers can't keep up.
Tier the gates: auto-approve classes with proven low error, keep humans on high-value and novel cases, and route by approver expertise. Don't let the queue grow — a 2-day approval backlog breaks the product.
3What happens on reject?
The reason goes back to the agent as an observation; it re-plans once, and a second rejection ends the run with a handoff to a person. Rejection reasons are clustered weekly into eval tasks.
Red flags interviewers listen for
- Gates every step.
- Shows the reviewer the raw transcript instead of a decision-ready summary.
- Never revisits gates based on measured approval and edit rates.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Gate selection | Risk classes named; fraud-prone actions never auto-approved. |
| Reviewer UX | Decision-ready card with diff, evidence and reason. |
| Evidence | Relaxation criteria with sample size and error bound. |
| Org scope | Policy, staffing and infrastructure owners; segregation of duties. |
Sources
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog
- Temporal documentation — Temporal Technologies, 2025docs