Design a deep-research agent that answers open questions with a cited, multi-page report.
- 5 min — scope: question types, depth, latency tolerance, price point
- 10 min — planner, sub-agents, notes store, writer
- 10 min — citation verification and injection containment
- 10 min — evals, budgets, cost per task
- 5 min — ownership, build vs buy for search
Use orchestrator-worker multi-agent: a lead agent plans sub-questions and spawns parallel sub-agents, each running its own search → read → note loop in a separate context window; a writer synthesises a report whose citations are checked against fetched text. Parallelism buys breadth and wall-clock time; the price is tokens. Anthropic's June 2025 write-up of its research system reported multi-agent runs using roughly 15× the tokens of a chat interaction — worth it only for high-value questions.
Question: 'Compare EU, US and UAE rules on disclaimers for AI-generated medical advice.' The planner emits six sub-questions, each with a budget (≤15 tool calls, ≤150k tokens). Sub-agents call web and internal search, fetch pages, strip boilerplate, and write notes as {URL, quoted span, claim} into a shared store — not into the lead's context. The writer drafts from notes only. A citation checker re-reads each cited span and drops claims the span does not support (fail closed). Each step is checkpointed with durable execution so a 10-minute run survives a pod restart.
- The eval set is the product. 200 research questions with rubric-graded reference answers; score factual accuracy, citation precision (does the span support the claim), coverage and source quality, using an LLM judge calibrated against domain experts.
- Failure modes: SEO spam and machine-generated pages as sources (domain-quality priors); prompt injection in fetched pages — sub-agents get read-only tools and no user data, which breaks the lethal trifecta; runaway fan-out (hard caps on agents and tokens); duplicated work (planner assigns disjoint scopes).
- Org scope: the platform owns the agent runtime, search/fetch connectors and budgets; product teams own prompts and per-domain evals. Buy search APIs; build citation verification — it is your liability boundary.
Assumptions: 10k research tasks/day; per task 1 lead + 5 sub-agents, 75 searches and 120 page fetches, ~1.2M input tokens (≈40% cache reads) and ~60k output including reasoning tokens (billed as output); illustrative prices $3/M input, $15/M output, $0.30/M cache reads; search API $5 per 1,000 queries; ~8 min per task.
- $ per task: uncached 720k × $3 = $2.16 + cached 480k × $0.30 = $0.14 + output 60k × $15 = $0.90 + search 75 × $0.005 = $0.38 ≈ $3.60.
- Monthly: 10k × 30 × $3.60 ≈ $1.08M/month. A chat answer at ~$0.01 is ~360× cheaper — gate by plan or quota.
- Concurrency: peak 3× average ≈ 0.35 tasks/s × 480 s ≈ 170 tasks ≈ 1,000 live agent loops.
- Fetch rate: 0.35 × 120 ≈ 42 pages/s → ~40 concurrent fetches at ~1 s each, plus per-domain politeness limits.
- Model rate: ~1,000 loops × one call per ~10 s ≈ 100 calls/s × ~13k tokens ≈ 80M input tokens/min — agree tokens-per-minute limits before launch.
What the answer sounds like at each level
A ReAct loop that searches the web and writes a summary with links.
Orchestrator-worker with parallel sub-agents, notes outside the lead context, token budgets and citation checking.
Prices the architecture per task, gates it by plan, builds an expert-calibrated eval set, and treats injection and citation precision as the trust boundary.
Follow-ups the interviewer will push
1Make it 10× cheaper.
Cheaper model for sub-agents and page summarisation, frontier model only to plan and write; cap pages per sub-agent by relevance; cache search results and fetched pages for 24 h across users; add a 'quick' mode with no sub-agents.
2An enterprise wants it over internal documents only.
Swap web search for enterprise search with per-user ACLs resolved at query time; sub-agents act as the user, never a service account; the report inherits the most restrictive ACL of its sources.
3A report cites a page that doesn't say what the report claims.
Citation precision failed: the checker must compare claim to quoted span (NLI model or judge), not just confirm the URL resolves. Add the case to the eval set and alert if daily-sample citation precision drops below 95%.
Red flags interviewers listen for
- A single agent stuffing every fetched page into one context window.
- Citations that are never verified against the source text.
- Sub-agents with write tools or access to user data while reading untrusted web pages.
- No per-task token or tool-call budget.
How it's scored
| Dimension | What strong looks like |
|---|---|
| Architecture | Parallel sub-agents with isolated contexts and a notes store; clear planner/writer split. |
| Trust | Span-level citation checking, read-only sub-agents, fail-closed on unsupported claims. |
| Economics | Per-task cost with token and search line items; gating by plan. |
| Evaluation | Expert-calibrated rubric set with citation precision as a first-class metric. |
Sources
- Building effective agents — Anthropic, 2024blog
- ReAct: Synergizing Reasoning and Acting in Language Models — Yao et al., 2022paper
- Not what you've signed up for: Compromising Real-World LLM-Integrated Applications with Indirect Prompt Injection — Greshake et al., 2023paper