Vol V · Section 3
Estimation & debugging drills
Back-of-envelope maths and production debugging scenarios, timed like the real round.
1Estimate the KV cache bytes per token for Llama-3.1-8B, and how many concurrent users one H100 80 GB can hold at a 4k-token average context.2Estimate tokens/s per GPU and the number of H100s needed to serve 50 QPS of a 70B model with 1,500-token prompts and 300-token answers.3Estimate $/1M tokens for self-hosting a 70B open-weights model versus calling an API, and the monthly volume at which self-hosting breaks even.4Estimate the time and cost to embed a 50M-chunk corpus, via an API and self-hosted.5Estimate vector index memory for 100M × 768-d vectors with HNSW, IVF-PQ, and binary quantisation.6How many eval examples do you need to detect a 3-point improvement in pass rate with confidence?7Debugging drill: p99 time-to-first-token jumped from 0.9 s to 4.5 s right after a deploy. Walk through it.8Debugging drill: after upgrading the LLM judge model, every system's faithfulness score rose 6 points overnight. What happened and what do you do?9Debugging drill: one tenant's agent runs are looping — hitting the 50-step cap and costing 30× normal. Walk through it.10Debugging drill: semantic cache hit rate fell from 35% to 4% over a day. Walk through it.