Trajectory evaluation
Evaluating an agent's whole path — tool choices, arguments, order, safety, efficiency — and its end state, over repeated trials (pass^k).
Checking only an agent's final answer misses a lot. It might be right by luck, take 40 steps where 4 would do, or do something unsafe on the way. Trajectory evaluation looks at the whole sequence of steps (the trajectory), not just where it ended.
It asks: did it use the right tools, in a sensible order, without wasting money or breaking rules — and does it do that every time, not just once?
Marking a maths exam on the working, not just the final number — and setting the same kind of problem three times to see if the method holds.
Requiring an exact match to one golden trajectory. Agents find different valid paths; assert required and forbidden steps and the end state instead.
Three levels. Outcome: task success checked against the final environment state — τ-bench (Yao et al., 2024) compares the database state at the end of a conversation with the annotated goal, which is more reliable than judging the final text; SWE-bench checks that hidden tests pass. Process: tool-selection accuracy, argument correctness, required steps present, forbidden actions absent, ordering constraints (e.g. verify_identity before change_payee). Efficiency and safety: steps, tokens, latency and $ per success; guardrail trips.
Methods: exact match to a reference path (brittle — many paths are valid), rule-based assertions over the trace, and LLM-as-judge with a Rubric over the transcript. Agents are stochastic, so run each task k times: pass@k = succeeds at least once (capability), pass^k = succeeds every time (reliability). τ-bench found gpt-4o under 50% pass^1 and under 25% pass^8 on retail. Build the suite from real failure traces via Error analysis and run it in an Eval harness with sandboxed or simulated tools.
100–300 tasks from real traces; tools replayed from fixtures or run in a sandbox; end-state check per task; trace assertions for required, forbidden and ordered steps; 4 trials per task reporting pass^1 and pass^4; cost and steps per success; run on every prompt, model or tool change as a CI gate.
Banking-support agent eval, 120 tasks × 4 trials. Final-answer correctness: 92%. Trajectory assertions show 18% of runs called change_payee without verify_identity first, and runs averaged 11 steps vs 5 on the reference. With the ordering assertion as a hard fail, pass^1 drops to 74% and pass^4 to 51% — the real reliability number, and the one that changed the launch decision.
Recommend end-state checks + trace assertions + k trials, and report cost per success. Say it plainly: "grade the destination, audit the route, run it more than once."
Expect: "how do you build the task set?" — start from production failure traces, add the policy-critical paths, keep 20% held out, and replay tools from recorded fixtures for determinism.
Numbers worth knowing
- pass^k with independent trials
- p^kp = 0.8 → pass^8 ≈ 0.17
- gpt-4o on τ-bench retail
- <50% pass^1, <25% pass^8Yao et al., June 2024
- Trials per task
- 4–8enough to separate pass^1 from pass^k at 100+ tasks
How to decide
| When | Do this |
|---|---|
| The task changes external state | Grade the end state, not the final message. |
| Policy requires an order (verify before act) | Hard trace assertion; any violation fails the run. |
| Users repeat the same task often | Report pass^k, not pass@k — reliability is what they feel. |
Cost maths
Eval cost = tasks × trials × cost per run. 200 tasks × 4 trials × $0.30 = $240 per full run; on every PR with ~40 PRs/month ≈ $9.6k. Cut it: a 50-task smoke set per PR (4 trials) ≈ $60 × 40 = $2.4k, full set nightly ≈ $7.2k → ≈ $9.6k total but with faster PR feedback; or cache tool fixtures and use a cheaper judge for process checks.
Operating it
- pass^1 − pass^4 gap exceeds 15 points (reliability, not capability, is the problem)
- Production failures not represented in the suite appear weekly
- Eval wall time blocks merges > 30 min
- pass^1 and pass^4 per release (alert on −3 points)
- Forbidden-action rate in production traces (alert on any)
- Cost per successful task p50/p95 (alert at +25%)
- Share of production failure modes covered by suite tasks (review monthly)
When it goes wrong
A newly launched account-servicing agent scored 92% offline but generated a stream of escalations in week one.
Escalation rate 4× forecast; trace review found payee changes made without identity verification.
Offline eval graded only the final reply text with a single trial; ordering violations and run-to-run variance were invisible.
End-state grading, ordering assertions as hard fails, and 4 trials per task; the corrected pass^4 (51%) led to tightening change_payee to require a verified-session flag in code.
Where it shows up
Sources
- τ-bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains — Yao et al., 2024paper
- SWE-bench: Can Language Models Resolve Real-World GitHub Issues? — Jimenez et al., 2023paper
- Judging LLM-as-a-Judge with MT-Bench and Chatbot Arena — Zheng et al., 2023paper
Check yourself
Three questions. Every option is explained once you answer.