Senior track
Running it in production
Latency, cost, rollouts, migrations and incidents — the operational layer most tutorials skip.
Senior · Running it in production
0 of 12 done · about 9.1 h totalStart: The real cost of an LLM feature: latency, tokens and the bill
- Read · 7 minThe real cost of an LLM feature: latency, tokens and the billFive terms make up the bill and two make up the wait. Reasoning tokens, cache writes, batch discounts and the self-host break-even, worked with stated assumptions.
- Learn a term · quiz · ~8 minKV cacheStored attention keys and values for past tokens, so each new token skips recomputing the sequence; at long context it fills GPU memory.
- Learn a term · quiz · ~8 minp50 / p95 / p99 latencyLatency at a percentile: p50 is the typical request, p95/p99 the slow tail. SLOs use the tail because averages hide the users who wait.
- Work through a section · ~2 hTraining, tuning & inferencePretraining, fine-tuning, RLHF, and the economics of serving a model.
- Read · 10 minWorkflows before agents: pick the simplest pattern that worksSeven patterns from Anthropic’s taxonomy, from one call to a fully autonomous AI agent, with the cost and latency spread of each and the failure that justifies climbing to the next.
- Learn a term · quiz · ~8 minDurable execution & checkpointingRunning agent workflows so progress is checkpointed and they resume after crashes, deploys or long waits without redoing completed side effects.
- Work through a section · ~2 hAgents in productionReliability, cost, evaluation, security and human oversight for agents that run for real.
- Read · 10 minShipping LLM features safelyEval gates on repeated runs with confidence intervals, versioned configs, canaries sized by arithmetic rather than habit, and a rollback you have rehearsed.
- Learn a term · quiz · ~8 minEval gate (CI)A CI check that runs the eval suite on every prompt, model or config change and blocks the merge or rollout if quality drops beyond the measured noise.
- Learn a term · quiz · ~8 minCanary & ramped rolloutRelease to a small slice of real traffic first (1%, then 5%, 25%…), compare against the old version, and roll back automatically if metrics degrade.
- Work through a section · ~2 hLLMOpsGateways, versioning, cost, rollouts, red teaming and model upgrades.
- Work through a section · ~2 hProduction RAG SystemsMulti-tenancy, security, caching, monitoring and incidents — judgment under constraint.