Describe the agent loop. How does it know when to stop?
- 2 min — loop mechanics
- 3 min — natural stops vs guard rails, with the submit-tool pattern
- 3 min — setting caps from the p99 of successful runs
- 2 min — degraded results and exit-share monitoring
- 2 min — budgets as platform policy
The loop: assemble context → call model → if it requested tools, execute and append results → repeat; if it produced a final answer, exit. Stop conditions are of two kinds. Natural: the model ends its turn without a tool call, or calls an explicit submit/finish tool with a schema (preferable — it makes “done” a structured event you can validate). Guard rails enforced in code: max steps, max input+output tokens or dollars, wall-clock timeout, a loop detector (same tool + same arguments ≥3 times, or no new information in N steps), and an error budget (e.g. 3 consecutive tool errors).
Set caps from data: take successful runs on the golden set, and put the step cap at ~1.5× their p99. If successful runs have p50 6 and p99 14 steps, cap at 20; anything past that is almost always a stuck loop.
Code-search agent answering “where is rate limiting enforced for the public API?” in a 2M-line monorepo. Budget: 20 steps, 150k cumulative input tokens, 90 s wall clock. Loop detector trips if grep is called with an identical pattern twice. In production, 93% of runs end naturally at a median of 5 steps; 5% hit the token budget, 2% the loop detector. A guard-rail exit doesn't throw — it returns the files found so far, the hypotheses still open and status: "budget_exhausted", so the IDE shows partial results instead of an error.
Natural stops are the model's judgement; guard rails are yours — never rely on the first alone. Track three numbers per release: steps-per-task distribution, share of runs ending on each guard rail, and success rate of guard-rail exits. A guard-rail share moving from 7% to 12% after a model or prompt change is an early drift signal that shows up days before user complaints. Make every exit path produce a useful, degraded result, and make the final answer pass a validator (schema, citations present) before the loop is allowed to end.
Org scope. Budgets are policy, not per-team constants: the platform runtime enforces a default (e.g. 25 steps / $0.50 / 120 s) and teams request overrides with data. Finance sees spend by guard-rail exit type. Build vs buy: every agent SDK has max_turns; few give you loop detection, dollar budgets and partial-result contracts — those you build once centrally. Failure mode: caps set by guess — too low truncates legitimate long tasks, too high lets a stuck loop burn $5 per run.
What the answer sounds like at each level
Loop until the model answers without calling a tool, with a max-steps limit as a safety net.
Separates natural stops from code-enforced guard rails, sets caps from the success distribution, detects loops, and returns partial results on budget exits.
Turns budgets into platform policy with override-by-evidence, monitors guard-rail exit share as a drift signal, and ties exits to cost attribution.
Follow-ups the interviewer will push
1How do you detect a loop that isn't an exact repeat?
Hash the (tool, normalised arguments) pair and also track progress: if N consecutive steps add no new entity ids, files or facts to the scratchpad, treat it as stalled. A cheap judge call every 10 steps asking “is progress being made?” is a fallback, not the primary signal.
2Now the model gets 2× faster but takes more steps. Do budgets change?
Wall-clock caps loosen, step and dollar caps don't. Re-derive caps from the new success distribution on the golden set; a model that needs more steps for the same success costs more per successful task even if each step is faster.
3Should the agent be able to ask for more budget?
Yes, via an explicit request_extension(reason) tool that pauses for a human or a policy check — never by silently continuing. It turns a hard cap into an auditable decision.
Red flags interviewers listen for
- Relies only on the model deciding it is done.
- Picks a step cap with no reference to the distribution of successful runs.
- A guard-rail exit throws an error and discards all work done so far.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Mechanics | Correct loop with explicit submit tool and validated final answer. |
| Guard rails | Steps, tokens/dollars, wall clock, loop detection, error budget — derived from data. |
| Degradation | Every exit returns a useful partial result with a status. |
| Org scope | Central budget policy, overrides on evidence, exit share as a release metric. |
Sources
- ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022paper
- Building effective agents — Anthropic (Schluntz & Zhang), 2024blog