How Netflix Shapes Our Fleet for Efficiency and Reliability

Every infrastructure engineer has been asked to make a service cheaper and to make it never go down, usually in the same quarter. Joseph Lynch and Argha C open their talk by asking the audience to admit that this feels like an inherent tension, and then spend the rest of it arguing that the tension is mostly an artefact of measuring the wrong things. Their central thesis is that efficiency and reliability are two sides of one optimisation problem: balancing the supply of computers against the demand from workloads, with a deliberate, priced-in amount of risk in between. If you get the accounting right, you can be efficient and reliable; if you optimise CPU utilisation, you will get neither.

Joseph Lynch is a Principal Software Engineer at Netflix working on stateless and stateful infrastructure, and covers the talk from services down to datastores. Argha C leads Netflix's Cloud Scalability efforts for Live, and covers clients down to services. InfoQ's 50:56 recording of the QCon San Francisco 2025 talk is dated May 5, 2026. These notes report what the speakers presented; where I add background that was not in the talk, it is labelled as such.

What You Will Learn

• Why "efficiency" defined as high CPU utilisation is actively harmful, and what Netflix uses instead.
• How to price the cost of failure into a capacity decision using risk-adjusted net value, and how service tiers make that concrete.
• What buffer means as a capacity configuration, why it differs between stateless and stateful services, and why slow-starting services legitimately run cold.
• How reserved versus on-demand pricing creates both a capacity risk window and a free work-relocation window.
• Why newest-generation instances are the most cost-efficient per dollar and simultaneously the riskiest to standardise on.
• Why microservice call graphs amplify a front-door traffic spike differently for each service, so you cannot extrapolate capacity linearly.
• The three nested control loops Netflix uses — hardware shaping over months, pre-scaling and autoscaling over minutes, hammers and prioritised load shedding over seconds.
• How prioritised load shedding lets errors rise without dropping successful requests per second.

Redefining Efficiency: Value, Cost, and the Loss Function

The talk starts by rebuilding the definition of efficiency from scratch, because the speakers believe most of the industry's confusion lives here. Workloads create business value — they do something useful. They also have a cost: you run them on computers and you pay humans to tend to them. Lynch is explicit that the talk will not tell you how to manage human cost ("I don't know how to do that"), only the compute cost of CPU, network, RAM, and disk, which is usually proportional to resource usage.

The insight that changes everything is that failures also have a cost. Lynch illustrates with an outside example: Amazon has, he believes, around $700 billion in revenue a year, which works out to roughly $20,000 for every second it is not accepting orders. He is careful to note Netflix's revenue is "not yet that big," so treat the figure as an illustration of scale rather than a Netflix number. The point is that downtime is a real line item, and if it is not in your capacity maths then your capacity maths is wrong.

Netflix models this formally. Failures of a service have some cost — modelled in the talk with a beta distribution, though Lynch says to pick your favourite — and they occur at some frequency, modelled as a Poisson process. Integrate the two and look at the tail of the resulting distribution, because the tail is where the incidents that get you tweeted about live. That gives a loss function per service, and from it:

efficiency = value − (cost + expected loss from failure)

Lynch calls the result risk-adjusted net value: a service has value, it has cost, and you are willing to spend money if that spend mitigates substantial business risk. A highly efficient service is one where value is significantly higher than the risk-adjusted cost; a highly inefficient one has those inverted. The word "utilisation" does not appear anywhere in that definition.

Different services have very different loss functions, and Netflix encodes this as tiers.

Lynch introduces these with "here are four, at Netflix we call them tiers," then enumerates only tier 0, tier 1, and tier 2. Because their loss functions differ so sharply, he says of a fleet running uniformly at 50% CPU, "I contend this is a very inefficient fleet shape." It makes no sense to spend the same amount of risk mitigation on personalisation as on playback. The fleet shape he wants is deliberately uneven: save dollars on the unimportant services and spend them paying down risk on the critical ones. "Playback needs capacity. Personalization, if we have it. If we don't have it, I'm willing to sacrifice it to ensure business outcomes."

Why Utilization Is a Lie

Utilisation is one input to efficiency, not efficiency itself. Lynch grounds this in Kingman's formula, which approximates the mean waiting time in a general single-server queue. Its shape is the hockey stick everyone has lived through: as utilisation approaches 100%, queueing delay does not degrade gracefully, it explodes.

Supplementary background for readers who have not met it: Kingman's approximation for mean wait time is roughly

E[Wq] ≈ ( ρ / (1 − ρ) ) × ( (c_a² + c_s²) / 2 ) × E[S]

where ρ is utilisation, E[S] is mean service time, and c_a and c_s are the coefficients of variation of arrival intervals and service times. Reading that equation is the whole argument of this section, because it shows utilisation is only one of several inputs. Lynch's point is that in steady state, the other terms are the ones you should attack:

• Average service time. Make the service faster and it does more work for less money. There is an entire discipline for this — performance engineering.
• Variability of service time. A service whose latency is consistently fast can be run at a higher utilisation safely. An erratic one cannot.
• Variability of arrival rate. This, Lynch says, is why Netflix gives so many talks about load balancing. If you can take bunched-up load and spread it out, you can safely run the same hardware hotter.

He then shows three services all sitting at 30% CPU utilisation with completely different shapes, and notes that only one of them is safe to run hotter — "and it's definitely not the yellow one." The takeaway he offers as a replacement question is: where do we spend our dollars, and do those dollars mitigate risk? If they do not, pick one of the three non-utilisation levers to reduce cost.

The same critique applies to reliability. Netflix defines reliability as responding with low latency to the business domain a service serves, scheduling batches with low latency, and — crucially — failures that are rare, fast to recover, and low impact. Lynch argues you should stop measuring nines and start asking three questions: how often does it fail, what is the impact, and how long does it take to recover. He shows three services with identical nines whose fixes are completely different — one needs backpressure or load shedding, another needs targeted failover. Averages hide the shape of the distribution, and the shape is what determines the remedy.

Buffer: The Real Capacity Configuration

Instead of headroom expressed as "keep CPU under 60%," Netflix talks about buffer: the ratio over current offered load that a service can absorb.

• Success buffer — how much more traffic the service can accept and still respond successfully. A 2x success buffer means you can double traffic and be fine.
• Failure buffer — how far beyond that you can go while shedding load, without entering congestive failure. A 4x failure buffer means the service can shed requests up to 4x offered load and remain in a defined state.

Past the failure buffer, behaviour is undefined and you may enter congestive failure, where the system is spending so much of its capacity on queueing, retries, and timeouts that useful throughput collapses. Lynch's advice is simply to stay far away from it, and Argha's later section covers the levers for doing so.

Buffers can expand, thanks to the cloud, but not instantly — there is a lead time to inject capacity once a service enters its success buffer. That gives the formal definition Lynch offers: buffer is a function of three pieces of context — how important the service is, to which part of the business (those two produce the loss function), and how quickly the buffer can be recovered once consumed.

The consequence is worth stating plainly, because it inverts a common instinct: low utilisation is a trade-off, not automatically waste. A fast-starting service can run with less buffer because the risk of running hot is low. A slow-scaling stateful service that has to shuffle bytes around when it fails should run with more buffer, and that is a good engineering decision rather than laziness.

Buffer also differs by workload type in a way pure CPU numbers obscure. A datastore needs spare CPU for background work. Lynch asks the room how many people have had an incident caused by PostgreSQL autovacuum or Cassandra compaction, treating both as common failure modes for stateful services. His conclusion is blunt: "Do not heat your Cassandra database. You're taking systemic risk... you're picking up pennies in front of a steamroller." If you look at a hot Java stateless service and a cooler Cassandra node and conclude the Java service is more efficient, you have missed the point entirely.

Understanding Hardware Supply

Having defined the target, the talk turns to the supply side: how do you actually acquire computers, and what does that cost in risk?

Reserved Versus On-Demand

The simplest way to guarantee capacity is to reserve it — pay the cloud provider in advance in exchange for a guarantee, with an asterisk on the word guarantee. Netflix's slow-scaling services run mostly on reserved hardware, which is efficient for them; the fast-scaling services mostly do not. Lynch stresses that reserving your entire need is not maximally efficient. The optimal reservation level sits somewhere between the trough and the peak of your demand curve, and where exactly depends on the ratio of reserved to on-demand pricing and on how long you expect to need the capacity.

That choice creates two structural consequences worth internalising:

1. During the on-demand phase you are carrying capacity risk. There is no guarantee the computers will be there when you ask, because you are not already running on them.
2. During the reservation trough you have free capacity. When demand sits below what you have already paid for, you have an opportunity to relocate work into that gap at effectively zero marginal cost.

The Generation Trade-off

Newer instance generations are more cost-efficient per dollar. They are also less available, because — as Lynch puts it — "the cloud's not actually real. Somebody has to actually rack and stack computers, and that takes time." Standardising your entire fleet on the newest generation with no fallback is a systemic risk. The opposite end of the curve is also risky: securing large quantities of old-generation hardware is similarly hard, because the provider is retiring it. Netflix therefore thinks about how the shape of its fleet distributes across generations and evolves that distribution as providers evolve.

Supply also varies over time and by instance shape. If you turn up to your cloud provider on Black Friday and ask for a lot of computers, so is everyone else. If you must do it for a business reason, the tools available are reservations, on-demand capacity reservations, previously held buffers, and — a favourite of Lynch's — preferential allocation, where infrastructure decides which workloads get the scarce machines. He notes the per-shape availability data he shows is illustrative rather than real, because providers do not publish it, but the shape of the problem is genuine: availability of any specific instance type is highly time-variable.

The conclusion is not to fear the cloud. It is that flexibility is capacity. If your workloads can run in more places, you can access more capacity, save money, and improve reliability. Lynch frames it as the next step in an old migration: the industry stopped naming individual servers and moved to cattle; now it needs to stop naming the particular instance type it wants and start naming the shape it wants.

Instances Are Not Interchangeable

Making workloads fungible runs into a real obstacle: computers differ in ways that are not captured by vCPU count. Taking a workload from an m6i to an m7a means:

• m7a cores are not Hyper-Threaded, so those vCPUs are roughly twice as effective as the vCPUs on the Intel instances. (Supplementary background: on a Hyper-Threaded core two hardware threads share one physical core's execution resources, so a vCPU there is a fraction of a core rather than a whole one.)
• Instructions per clock and clock frequency differ between the architectures.

Netflix spent significant time on this and has open-sourced a library that provides an apples-to-apples comparison, letting them predict whether a workload moving between shapes will need more cores, fewer cores, or will not work at all. Feed three things into a model — shape data, workload context, and the reserved-versus-on-demand price differences — and it outputs the cost-optimal computer. Lynch is careful here: "most models are wrong, but sometimes they're useful." It is not truly the optimal machine for any single workload, but applied across the whole fleet it produces good aggregate efficiency and reliability. The same machinery supports planning under uncertainty: given a demand range, you can compute regret functions and pick the instance you will least regret across the most scenarios.

One non-obvious result falls out of the maths. Cloud providers price roughly linearly in size, but efficiency is not linear in size. Larger instances can be run hotter for the same buffer. Lynch attributes this to the probability that an arriving job finds a non-busy core being better on larger machines, though he hedges the direction of the effect out loud; the supplementary framing is that this is the familiar queueing result that pooling more servers into one queue reduces waiting time at a given utilisation. So why not run the entire fleet on m7a.32xlarge? Because finding many contiguous cores is harder for a provider than finding the same number of fragmented ones, exactly as finding contiguous memory is harder for an allocator. Moving to bigger, more efficient hardware buys efficiency at the price of acquisition risk. That is the efficiency-reliability tension in one graph.

The supply-side model therefore takes each workload's demand curve — long-running and variable, stable, or needed for only a few minutes — and optimises jointly for financial cost and capacity guarantee. Depending on the risk it might decline to reserve while still issuing an on-demand capacity reservation, or reserve some number between projected demand, minimum, and peak.

Lynch's four takeaways for this half: buffer is the relevant configuration for efficiency, not CPU utilisation; capacity must be prioritised by risk; time to recover matters deeply to fleet efficiency; and with the right maths, compute is highly fungible, which is what lets you mitigate capacity risk.

Understanding Software Demand

Argha takes over for the demand side, which starts at the individual workload and zooms out to the global fleet.

Workload Profiles

A workload profile is the resource footprint a service needs to do its work, across three dimensions.

The memory point deserves unpacking for readers who do not tune JVMs daily. Two services with identical live heaps can behave completely differently if one allocates short-lived objects far faster: the collector runs more often, consumes more CPU, and needs more headroom in the young generation. Whether the collector paces itself against allocation rate or not changes how much slack you must leave. Sizing on live heap alone will under-provision the high-allocation service.

Scaling Behaviour and Startup Time

To turn a profile into capacity, you need to know how the service scales. The simplest technique Argha recommends is empirical: observe the workload in production, and fit a correlation between CPU and request rate — request rate being a proxy for the amount of work the service is doing. That fit yields scaling targets. He flags one simplification openly: this assumes you already load balance fairly.

The component he says is most often missed is startup time. It is not enough to say "I need compute, schedule it, done." If your service is slow to start, that by definition means you must hold more buffer to cover the gap. And startup time regresses in practice — a library upgrade introduces a slowdown, and the fleet has to compensate for it in capacity terms until it is fixed. This is the demand-side mirror of Lynch's "time to recover matters" takeaway.

Architecture And Data Flow

Argha presents a deliberately simplified view of the Netflix streaming control plane, which matters because the topology determines how a traffic spike propagates.

flowchart LR
    Clients["Global clients
mobile, PC, TV"] --> Zuul["Zuul
open-source API gateway (front door)"]
    Zuul --> Fed["Federation layer"]
    Fed --> Playback["Playback service
tier 0"]
    Fed --> Discovery["Discovery / browse"]
    Fed --> Personal["Personalization"]
    Playback --> DG["Data gateways"]
    Discovery --> DG
    Personal --> DG
    DG --> Stores[("Datastores")]
    Clients -.->|"video bytes, not control plane"| OC["Open Connect CDN"]

Netflix runs this control plane fully active across four AWS regions, paired with Open Connect, its own CDN, which delivers the playback bytes from as close to the viewer as possible. The control plane gets you to press play; the CDN delivers the video.

The critical observation is that a microservice call graph amplifies traffic non-uniformly. In the example Argha shows, a 4x load spike arriving at the front door means:

You cannot linearly extrapolate a front-door spike across the fleet and provision against it. Each service's amplification factor is a property of the call graph and has to be measured.

Two further global properties shape demand. First, regional peaks are temporally distributed: within a single region the delta between peak and trough is around 10x. Provisioning every region for its own peak would be wildly inefficient at Netflix's scale, so they explicitly do not buy for peak. The upside of the diurnal pattern is predictability, which is what makes failover planning possible. Argha's example: during a recent AWS us-east-1 outage, Netflix did not trend on Twitter, because the learned response is to get out of the troubled region. He does not spell out the dependency, but the implication is that evacuating a region is only viable if the remaining regions have room — which is what the predictable diurnal pattern lets you verify in advance.

Second, some demand is genuinely unpredictable. A major content launch — he cites new seasons of The Witcher and Stranger Things — produces binge-watching spikes that are great for viewers and hard to forecast precisely.

Balancing Supply and Demand

Argha frames the balancing act with appropriate scepticism: being asked to make supply and demand fit perfectly is like being asked to predict the future or time the market. Before the mechanisms, he states three tenets.

1. Locally efficient decisions can add global risk. A service that provisions enormous amounts of the most performant hardware may be locally optimal while consuming, say, 40% of the available performant capacity and starving critical services. Optimise at the fleet level, not per team.
2. Buffers must exist where and when they matter most. For the most critical services, capacity has to be available at exactly the moments they need to scale out.
3. Infrastructure must do the heavy lifting. In an organisation with thousands of engineers and teams, a solution that requires every service owner to get this right does not scale and arguably cannot.

The mechanisms that follow form three nested control loops operating at completely different timescales, reacting to different context, and running independently.

flowchart TD
    subgraph Months["Months — manage supply"]
        Shape["Hardware shaping
preferred + fallback instance types"]
        Reserve["Reservations and
capacity negotiations"]
    end
    subgraph Minutes["Minutes — manage demand"]
        Pre["Pre-scaling / min pinning
from viewership forecasts"]
        Steer["Traffic shaping + DNS steering
pre-ingress"]
        Auto["Reactive autoscaling
target tracking"]
    end
    subgraph Seconds["Seconds — last resort"]
        Hammer["Hammers
emergency capacity injection"]
        Shed["Prioritized load shedding"]
    end
    Months --> Minutes --> Seconds

Months: Managing Supply Through Shaping

Netflix builds capacity models — at minimum one for stateful and one for stateless workloads, because they behave very differently. Each model takes two kinds of input: supply context (how much compute the provider or data centre actually has) and hardware specs (clock frequency, hyper-threading, cores, memory), collectively the shapes.

The per-workload output is a recommendation such as: this stateless workload should run on a c7a.2xl. That implies the model inferred a CPU-bound workload — a memory-bound one would have been steered to an r7a or m7a. Pricing data is folded in, so "optimal" means both performant and price-efficient for Netflix.

Two additions make it usable in reality. First, the model emits fallback types, because the preferred instance will sometimes be unavailable. Second, recommendations are validated by squeeze tests — actually running the workload on the hardware and recording the result, rather than trusting the model.

Driving these recommendations across the fleet is what Netflix calls shaping. Pricing, capacity, business context, and app tier feed in; out comes a rough capacity demand broken down per region and per instance type. Reality then intrudes: in Argha's example the model wants around 10,000 more m7a.xl instances than exist in the pool. The response is reallocation under a risk-minimisation constraint. In the same picture, r7a and m6i — a different generation and a different vendor — have healthy pool depth, which enables two distinct moves:

• Downgrade generation for non-critical services. Move less critical, non-tier-0 workloads to older, less performant, Hyper-Threaded Intel m6i capacity. Tier 0 is left alone as far as possible.
• Strictly safe lateral fallbacks. m7a to r7a is a safe migration if the app is memory bound — and the model can identify exactly which workloads qualify.

This loop runs continuously across all four regions, but it moves in months, not days. Pricing negotiations, context propagation, and coordinating migrations with service owners are all slow. That latency is precisely why the other two loops exist.

Minutes: Pre-scaling and Traffic Shaping

The first demand-side lever is the simplest: pre-scale the fleet ahead of a known event. Netflix runs prediction models — Argha calls them a crystal ball — that estimate viewership for a launch or live event, convert that into global traffic, then into per-region traffic, then into a number of computers. Crucially this is automated at fleet scale; it does not work as a per-service manual exercise.

In practice pre-scaling means min pinning: raise a service's minimum instance count from, say, 25 to 300, then let traffic arrive. Argha points out something easy to miss on the graph — there is still incremental autoscaling on top of the pin, and that is deliberate. They do not pre-scale so high that autoscaling never engages, because that would be wasteful. Afterwards everything scales back down gradually.

Pre-scaling quality has three regimes:

A real efficient pre-scale, he shows, includes a brief window of transient, minimal load shedding just before capacity lands. That is acceptable precisely because it is transient — and it is followed by RPS normalisation as the new instances come into rotation and traffic redistributes.

The next lever operates before traffic ever reaches a data centre, which Argha calls dynamic traffic shaping. It has two halves.

Redistributing existing traffic. Without shaping, regional traffic curves diverge sharply — he points to eu-west and us-west-2 pulling far apart — and wide variance means you cannot forecast compute demand with confidence. With shaping applied, the production graphs show the four regions tracking much more tightly, which directly improves how precisely they can provision.

Steering new traffic. Netflix runs authoritative DNS resolvers and controls DNS tiering, so it can decide which region a new session lands in. (Argha credits a separate talk by Sergey for the detail.) By default, steering optimises for latency, so you play a title as fast as possible — but the latency-optimal answer leaves us-east-1 running much hotter than the other regions. So they override it: traffic that would have hit us-east-1 is moved to us-west-2 and us-east-2, where capacity exists. During a large event this was the difference between balanced demand and one region — us-east-2, he thinks — running short of capacity, with direct streaming impact for viewers.

Seconds: Autoscaling, Hammers, and Prioritized Shedding

"Hope is not a strategy." When pre-scaling and steering are not enough, three reactive numbers govern each critical service, all derived mathematically from its buffer:

1. Target tracking — the CPU percentage at which the service begins scaling out. Controlled, incremental capacity addition.
2. RPS target — the same idea keyed on request rate rather than CPU.
3. Hammers — emergency capacity injection, triggered when the service is pushing into the right-hand end of its success buffer, i.e. one step away from having to shed. Netflix runs hammers on both CPU and RPS.

In operation these fire automated scaling events that raise minimums, provision instances, and — symmetrically — release the compute once traffic subsides. This compresses the fleet-scaling feedback loop from months to minutes.

Autoscaling still has limits; Argha notes it can be slow for reasons largely outside their control and credits a separate talk by Ryan for the detail. (Lynch similarly points to an earlier, maths-heavy talk of his own covering the hardware-matching model.) Netflix invested in this directly, and the reported result is concrete: a tier 0 service that used to take 8 to 15 minutes to recover from a 10x spike now takes three to four minutes, roughly a 70% reduction, measured in production over a year of work.

When even that is not fast enough, the last lever is load shedding. Argha distinguishes two modes. Undiscriminated shedding — which he also calls bulk shedding — drops traffic without regard to type. Prioritised load shedding instead sheds in order of business criticality: the least critical class of traffic first (confusingly, Netflix also calls that class "bulk"), then best-effort and degradable traffic, and critical traffic last, if at all. His analogy is letting an ambulance through on the highway while the sports cars wait.

The production result is the most quotable metric in the talk: non-critical shedding starts much earlier and well before critical shedding is touched, so while the error rate rises, successful RPS does not drop. Load was shed without compromising the service's effective capacity for the requests that mattered. Combined with hammers, this brings the innermost feedback loop down to seconds.

Containers, EC2, and Fungibility

The Q&A drew out an architectural point the main talk skipped. Netflix uses both EC2 instances and containers — the containers run on EC2 underneath. The allocation between them is deliberate:

• Direct EC2 is a much better fit for critical, latency-sensitive workloads that need very consistent performance. Lynch calls this an important learning from the project.
• Multi-tenant container platforms carry batch and less critical workloads, and are the main source of fleet fungibility, because capacity can be reclaimed from container pools and reallocated into the latency-sensitive EC2 pools.

The same shape model that recommends instance types also right-sizes container footprints. Lynch's example of the problem it solves is familiar to anyone who has run a container platform: a team requests 80 GB of RAM, the platform grants 80 GB, observability shows they use 10 GB, and someone has to go and ask whether they might consider not doing that.

On bin packing, Argha's view is that it is hard regardless — your provider is also bin packing underneath you — but that buying fixed sizes yourself reduces the capacity problem, and holding onto instances lets you exploit the peak-to-trough gap. The thing to optimise is fungibility: a single held instance is easier to make fungible than the same capacity scattered across many small ones. He also flags preemption as an advanced lever they did not have time to cover: combine criticality tiers with preemption and you can evict non-critical workloads to schedule critical ones, which is where a lot of the value comes from.

Trade-offs And Limitations

Beyond the tensions already covered — efficient hardware versus acquisition risk, reserved price versus capacity guarantee, and the fact that the supply loop moves in months — the talk names several limits on its own applicability.

• Prediction is imperfect, so some shedding is normal. Content launches cannot be forecast with the precision of diurnal patterns, and even a well-executed pre-scale shows transient shedding before capacity lands. The engineering judgement is that brief, non-critical shedding is cheaper than the buffer needed to eliminate it.
• Autoscaling depends on things you do not control. Provider scheduling and service startup time both bound how fast it can be, and startup time silently regresses with dependency upgrades.
• Raw compute does not buy down risk on its own. This is Argha's second closing lesson: the problem needs holistic solutions and systems thinking, because efficiency at scale is "a compounding game." Buying more instances without the tiering, buffer, and traffic work does not get you there.
• Not all of this is portable. Argha is candid that he is "very opinionated" and "spoiled by Netflix," and that much of the pre-ingress work is impossible without end-to-end control over your traffic — including authoritative DNS. If your traffic arrives through a partner or a CDN you do not control, the steering half of this playbook is unavailable to you.
• Human cost is out of scope. Lynch says outright that the talk covers compute cost only, even though he defines efficiency to include the cost of paying humans to tend to workloads.
• The models are wrong, deliberately. "Most models are wrong, but sometimes they're useful." The recommendation is not optimal per workload; the claim is that fleet-wide application produces good aggregate outcomes. Applying this to a handful of services would not obviously pay off.

Practical Takeaways

• Assign every service a tier with an explicit loss function before doing any capacity work. Tier 0 with no fallback, tier 1 degradable, tier 2 best effort is a workable starting taxonomy. Without it, "efficiency" has no denominator.
• Report buffer per tier instead of fleet-wide average CPU. For each service, track how much traffic multiple it can absorb successfully and how long it takes to restore that headroom once consumed.
• Attack the non-utilisation terms first. Reducing mean service time, reducing latency variance, and smoothing arrival rate through load balancing all buy you the ability to run hotter safely. They are usually cheaper than buying more instances.
• Measure your call-graph amplification factors. Replay or model a front-door spike and record what multiple each downstream service actually sees. Capacity plans built on a uniform multiplier will over-provision some services and leave others exposed.
• Never provision stateful services on the same utilisation targets as stateless ones. Reserve explicit background headroom for compaction, autovacuum, and repair.
• Build a fallback instance list, and validate it with squeeze tests. The fallback you have never run is not a fallback. Pay attention to whether a candidate is Hyper-Threaded, since vCPU counts are not comparable across vendors.
• Track service startup time as a first-class SLI. It directly sets the minimum buffer you must carry, and it regresses quietly with dependency upgrades.
• Build proactive and reactive levers, not one or the other. Argha's mental model is pre-ingress (steering, shaping, pre-scaling) versus post-ingress (autoscaling, hammers, shedding). Each has a floor on how fast it can respond.
• If you implement only one reactive lever, make it prioritised load shedding. It converts a total outage into a partial degradation of the least important traffic, and it is what lets successful RPS hold flat while errors rise.
• Exploit your reservation trough. Any period where committed capacity exceeds demand is free compute for batch, migrations, or preemptible work.

Key Terms

• Risk-adjusted net value — Netflix's efficiency definition: a service's business value minus its running cost minus the expected business loss from its failures. That last term is the loss function, and it is what separates a tier 0 service from a tier 2 one.
• Buffer — The multiple of current offered load a service can absorb. The success buffer is what it can serve successfully; the failure buffer is how far past that it can shed load without entering congestive failure.
• Congestive failure — The undefined regime past the failure buffer, where queueing, retries, and timeouts consume the capacity that should be serving requests and useful throughput collapses.
• Shape and shaping — A shape describes an instance type by its actual characteristics (cores, hyper-threading, clock frequency, memory) rather than its name, so workloads can be matched to any equivalent machine. Shaping is driving preferred and fallback instance-type recommendations across the whole fleet, per region and per instance type, as pricing and supply change.
• Squeeze test — Running a real workload on candidate hardware under increasing load to validate a model's recommendation empirically.
• Min pinning — Pre-scaling by raising a service's minimum instance count ahead of a forecast event, while leaving autoscaling active above it.
• Hammer — An emergency autoscaling trigger that injects capacity when a service is at the far end of its success buffer, before shedding becomes necessary. Netflix runs them on both CPU and RPS.
• Prioritized load shedding — Dropping traffic in order of business criticality, least critical first and critical last, rather than indiscriminately.
• Fungibility — The property of being able to run a workload on many different machines or reclaim capacity between pools; the thing that converts modelling effort into both savings and reliability.

The through-line of the talk is that Netflix did not resolve the efficiency-reliability tension by choosing a side. They made the cost of failure an explicit term in the same equation as the cost of compute, which turns an argument about values into an arithmetic problem — Argha's closing line is that maths is their safety blanket, and that combining it with some Econ 101 solves surprisingly hard problems. Most of the specific mechanisms here need Netflix's scale to pay for themselves. The accounting change does not.

Reference: Joseph Lynch and Argha C, How Netflix Shapes our Fleet for Efficiency and Reliability, QCon San Francisco 2025, published by InfoQ on May 5, 2026.