Reliability is undeniably the most fundamental feature of any software system. A system that is highly performant, beautifully designed, and feature-rich provides exactly zero value to the customer when it is offline. However, achieving absolute, 100% reliability is mathematically impossible and economically ruinous. To balance feature velocity against the risks of instability, modern engineering organizations—pioneered by Google's Site Reliability Engineering (SRE) practices—use a tiered framework of indicators, objectives, and agreements.
This deep dive covers the technical implementation, mathematical modeling, and business realities of Service Level Indicators (SLIs), Service Level Objectives (SLOs), and Service Level Agreements (SLAs).
An SLI is a carefully chosen quantitative measure of some aspect of the level of service that is provided to a customer. It is the raw data, the telemetry that informs all subsequent decisions. You cannot have an objective or an agreement without a precise, mathematically rigorous indicator.
When designing SLIs, engineers typically look to the "Four Golden Signals" as defined by the SRE handbook:
A common architectural gotcha is where to measure the SLI. If your API gateway logs show a 99.99% success rate, but user mobile applications are crashing due to a CDN misconfiguration, your users are experiencing an outage. The best SLIs are measured as close to the user as possible (e.g., client-side telemetry) to avoid blind spots in the network topology.
An SLO is a target value or range of values for a service level that is measured by an SLI. While the SLI is the measurement (e.g., "P99 latency is 350ms"), the SLO is the goal (e.g., "P99 latency should be less than 400ms for 99.9% of requests over a rolling 28-day window").
Availability is typically expressed in "nines." The difference between three nines and four nines is an order of magnitude in operational rigor, architectural complexity, and financial cost.
| Availability % | Nines | Downtime per Month (30 Days) | Downtime per Year | Architectural Reality |
|---|---|---|---|---|
| 99% | Two | 7.2 hours | 3.65 days | Standard for internal/non-critical tools. Single zone, single database. |
| 99.9% | Three | 43.8 minutes | 8.77 hours | Typical for high-quality SaaS products. Multi-zone redundancy required. |
| 99.95% | Three and a half | 21.9 minutes | 4.38 hours | Standard for critical enterprise services. Automated failover essential. |
| 99.99% | Four | 4.38 minutes | 52.6 minutes | "Gold Standard" — requires full automation, zero-downtime deployments, active-active replication. |
| 99.999% | Five | 26.3 seconds | 5.26 minutes | Global infrastructure (Carrier Grade). Extremely rare and expensive. |
The classical time-based availability calculation is:
However, in modern distributed systems, request-based availability is far more accurate than time-based availability. If your system goes down at 3:00 AM on a Sunday when traffic is near zero, the impact is fundamentally different than an outage at 11:00 AM on Cyber Monday. Request-based availability captures this reality:
The conceptual breakthrough of the SRE model is the Error Budget. An error budget is one minus the SLO. It represents the maximum allowable threshold for errors and outages in a specific measurement window. It mathematically aligns the typically opposing forces of product development (who want to ship features rapidly) and operations (who want to maintain stability).
Suppose you run an e-commerce platform that processes $10,000,000 in transactions over a 30-day window. You have established a 99.9% availability SLO. Your error budget allows for a 0.1% failure rate.
This $10,000 represents the maximum acceptable loss due to system instability. If the engineering team introduces a bug that causes a 2-hour outage during peak hours, and you lose $15,000 in revenue, you have blown your error budget.
When an error budget is depleted, there must be organizational consequences. The standard SRE practice dictates that if the error budget drops to zero, all feature deployments are halted. The engineering team must shift 100% of their velocity to reliability tasks (paying down technical debt, improving test coverage, optimizing database queries) until the rolling window recovers and the budget turns positive again.
Burn rate is the velocity at which you are consuming your error budget. It is the primary, most vital signal used for paging on-call engineers.
If your burn rate is exactly 1, you will consume 100% of your budget at the exact end of your rolling window. A burn rate of 1 is rarely a cause for alarm. However, elevated burn rates require immediate attention.
Consider a 30-day window with a 99.9% SLO (which allows a 0.1% error rate). If your system is currently experiencing a 1.44% error rate, you are consuming your budget 14.4 times faster than normal:
A burn rate of 14.4 means you will completely exhaust your 30-day error budget in slightly over 48 hours.
Alerting purely on a fast burn rate can result in flapping—pages triggering and resolving within minutes. Modern alerting strategies evaluate both a short window (e.g., 10 minutes) and a long window (e.g., 60 minutes) to confirm the burn rate is sustained.
An SLA is a formal, legally binding contract between a service provider and a customer. It dictates exactly what happens if the SLO is violated. SLAs are business instruments, not engineering instruments.
If you fail to meet your SLA, you owe your customer money. These are known as SLA credits or financial penalties. If a B2B SaaS platform has a 99.9% SLA and drops to 99.5% for the month, they might owe all affected customers a 10% credit on their monthly invoice.
Consider an enterprise contract worth $500,000 annually ($41,600 monthly). A typical penalty clause might look like this:
Because of the severe financial implications, SLAs are almost universally set significantly lower than SLOs. If your engineering SLO is 99.95%, your legal SLA should be no higher than 99.9%. The delta between your SLO and SLA serves as an operational buffer. If you violate your SLO, the engineering team is halted and fixes the issue before you violate the SLA and lose money.
You cannot arbitrarily promise four nines (99.99%) of SLA if your downstream dependencies do not support it. The composite availability of a system is the product of its critical dependencies.
If your web application depends on a database with a 99.95% SLA, a cache with a 99.99% SLA, and a third-party payment gateway with a 99.9% SLA, your maximum theoretical availability is:
In this scenario, it is mathematically impossible to offer a 99.9% SLA to your customers without implementing defensive architectural patterns like circuit breakers, asynchronous queuing, and graceful degradation (so that the system can remain partially functional even if the payment gateway fails).