Vol IV · Section 2
LLMOps
Gateways, versioning, cost, rollouts, red teaming and model upgrades.
1Design an LLM gateway for a company with many teams calling several model providers.2How do you handle rate limits and fallbacks across multiple model providers?3How do you version and roll out prompt changes safely?4How do you put eval gates into CI for an LLM application?5How do you A/B test an LLM change in production?6How do you measure and control cost per query for an LLM product?7A provider announces deprecation of the model you use. What is your upgrade playbook?8How do you approach red teaming and jailbreak defence for an LLM product?9How do you handle PII in an LLM system, from prompt to logs?10How do you monitor quality drift for an LLM feature in production?