SLO & SLA
Service Level Objective: a target for a measured indicator over a window, e.g. 99.5% of answers under 5 s over 30 days. Its gap is the error budget.
An SLO is a promise you make to yourself about how well a service should work, written as a number: for example, '99.5% of requests succeed within 5 seconds, measured over 30 days'. The thing you measure is the SLI (service level indicator). The 0.5% you are allowed to miss is the error budget. When it runs out, you slow feature releases and fix reliability.
A train operator promising that 95% of trains arrive within 5 minutes of schedule each month, and pausing timetable changes when they fall behind.
Setting the SLO at 100% or at today's observed performance. 100% leaves no room to ship; today's numbers may be far better or worse than what users need.
SLI = good events / valid events (e.g. requests that returned 2xx within 5 s / all non-client-error requests). SLO = target for that ratio over a rolling window. Error budget = 1 − SLO: 99.5% over 30 days allows 0.5% bad requests, equivalent to 3.6 hours of total outage. Alert on burn rate, not raw errors: the SRE Workbook's multi-window scheme pages when the 1-hour burn rate exceeds 14.4× (2% of a 30-day budget spent in an hour) and tickets at 6× over 6 hours (5%). LLM systems need quality SLIs alongside availability and latency: share of answers passing a faithfulness check, abstention rate within band, tool-call success rate. An SLA is the contractual version with penalties; keep internal SLOs tighter than any SLA. Set SLOs from what users notice, not from what the system happens to do today.
Per user-facing endpoint: availability SLO 99.5%, latency SLO on p95 total and TTFT, one sampled quality SLI; 28- or 30-day rolling window; multi-window burn-rate alerts (14.4×/1 h + 5 min page, 6×/6 h + 30 min page, 1×/3 d ticket); written error-budget policy agreed with product.
Developer-docs assistant. SLIs and SLOs over 28 days: availability 99.5% (non-5xx); latency 95% of answers complete in < 6 s and TTFT < 1.2 s; quality 97% of sampled answers with faithfulness ≥ 0.8. A provider incident burns 30% of the availability budget in 3 hours; burn rate hits 14.4× and pages. With 40% budget left mid-cycle, the team freezes a risky model swap until the window resets.
Say: I define three SLIs for an LLM endpoint (availability, latency per stage including TTFT, and a sampled quality SLI), set targets from user-visible thresholds, and alert on multi-window burn rates. Expect: 'the provider is down, does that burn your budget?' Yes; users do not care whose fault it is, which is the argument for a fallback model.
| SLO (30 d) | Error budget | Full-outage equivalent |
|---|---|---|
| 99% | 1% | 7.2 h |
| 99.5% | 0.5% | 3.6 h |
| 99.9% | 0.1% | 43 min |
| 99.95% | 0.05% | 22 min |
Numbers worth knowing
- 99.5% monthly budget
- 3.6 hours30-day window, full-outage equivalent
- Fast-burn page threshold
- 14.4× over 1 hconsumes 2% of a 30-day budget (SRE Workbook)
- Slow-burn threshold
- 6× over 6 hconsumes 5% of the budget
How to decide
| When | Do this |
|---|---|
| Single LLM provider dependency | Do not promise more than its availability; add a fallback model to go higher |
| Quality matters as much as uptime | Add a sampled faithfulness or task-success SLI with its own budget |
| Budget exhausted | Freeze non-reliability launches until the window recovers, per written policy |
Cost maths
Each nine costs more redundancy. Adding a fallback provider: ~0 fixed cost, but fallback answers may cost 1.5× and need their own eval. Going from 99.5% to 99.9% typically means multi-region retrieval replicas (2× index cost, e.g. $3k → $6k/month) plus a second LLM provider integration.
Operating it
- Budget exhausted two cycles in a row
- Enterprise customer asks for an SLA above your current SLO
- A single dependency causes > 50% of budget burn
- Error-budget remaining per SLO; review weekly
- Multi-window burn-rate alerts: page at 14.4×/1 h, ticket at 6×/6 h
- Quality SLI (sampled faithfulness pass rate); alert < 97% over 24 h
When it goes wrong
The latency SLO showed green all quarter while users complained of slow answers
Product analytics showed rising abandonment on answers over 8 s; the SLI excluded streaming requests
The SLI measured time to HTTP response headers, which the streaming endpoint sent immediately, so every streamed request counted as fast
Redefined SLIs as TTFT and time-to-last-token from the client's perspective; actual compliance was 91%, triggering a reliability sprint
Related terms
Where it shows up
Sources
- Site Reliability Engineering, ch. 4: Service Level Objectives — Google, 2016book
- The Site Reliability Workbook: Alerting on SLOs — Google, 2018book
Check yourself
Three questions. Every option is explained once you answer.