Vol V · Section 1
AI system design
Design ChatGPT-style serving, an eval platform, a coding agent, enterprise search — with numbers.
1Design ChatGPT-style chat serving for 10M daily users. How many GPUs do you need, and how do you get from QPS to that number?2Design an LLM evaluation platform used by 30 product teams.3Design a coding agent that takes a ticket, works in the company's repositories, and opens a pull request with passing tests.4Design a deep-research agent that answers open questions with a cited, multi-page report.5Design enterprise search with AI answers across Google Drive, Slack, Confluence, Jira and Salesforce, respecting each tool's permissions.6Design an LLM-based content-moderation pipeline for a platform receiving 200M posts and comments per day.7Design a customer-support copilot that resolves what it can and hands off to a human agent cleanly when it can't.8Design a semantic cache for a multi-tenant LLM product. How do you keep it correct?9Design a pipeline that extracts structured data from millions of PDFs a month — invoices, insurance claims, financial filings.10Design a realtime voice assistant. Walk through the latency budget from the end of user speech to the first audio back.