Service-Level Prioritized Load Shedding at Netflix: Buffers, Priorities, and Automated Validation

Autoscaling is the reflexive answer to "what happens when traffic exceeds capacity?" Anirudh Mendiratta and Benjamin Fedorka argue it is the wrong answer for the spikes that actually hurt: reactive autoscaling takes minutes, provisioning for the true peak is expensive, and sometimes the cloud has no instances to give you. Their thesis is that a service should degrade on purpose along an axis the business cares about, and that this capability should be generated and validated by a platform rather than hand-tuned per team.

Mendiratta is an engineer on Netflix's Playback Lifecycle team who worked on launching live streaming. Fedorka is an engineer on Netflix's Java Platform focused on IPC ergonomics and resilience. They gave this 50-minute talk at QCon San Francisco 2025; InfoQ published the recording and transcript on July 2, 2026. These notes report what the speakers presented; added context is labeled.

What You Will Learn

• Why scaling up is not a sufficient answer to legitimate traffic spikes.
• What congestive failure is, and why a cluster in that state does not recover.
• How the "success buffer" and "failure buffer" vocabulary turns overload into something you can set goals against and test.
• Why prioritized load shedding beats both equal-opportunity shedding and sharding critical traffic into its own cluster.
• How Netflix determines priority cheaply, propagates it through a call graph, and refuses to let services upgrade it.
• How utilization reaches an Envoy sidecar via ORCA files, and how a shedding function maps (utilization, priority) to a drop probability.
• How chaos experiments validate both priority labels and generated configuration before it reaches production.
• Why retries make overload worse, and what a prioritized attempt budget does.

The Problem: Spikes You Cannot Provision For

Netflix's Play API is called every time a member hits play. Its traffic peaks around 7 p.m. US-Pacific and troughs around 3 to 4 a.m., and on top of that predictable curve a title launch or live event produces a large spike. The team knows when these spikes happen but not how large they will be, and the traffic is entirely legitimate — this is not a DDoS problem, so a blocklist is not the answer.

Netflix does scale, both reactively (autoscaling on a resource measure such as CPU) and proactively (ahead of a known live event), but neither suffices. Reactive scaling takes minutes in the cloud: detect the need, request instances, boot them, pass health checks. Proactive scaling to the hypothetical peak, where every user joins at once, is expensive, and your provider may not have capacity when you ask. There is also the outsized thundering herd: during a live title many users rebuffer at the same moment and everyone hits refresh simultaneously, a shape you cannot extrapolate from ordinary peaks.

Load shedding is the alternative — deliberately reject excess traffic to keep latency low for what you do serve. Mendiratta distinguishes it from rate limiting: shedding concerns total volume against total provisioned capacity, while rate limiting enforces a fixed per-user limit and can also serve monetization goals such as API quotas. Good rate limiting reduces how often you must shed, but the two are not interchangeable.

Congestive Failure

Without shedding, ramping traffic into an instance produces a recognizable signature: latency climbs for all requests, p50 and p99 alike. For a viewer that is the spinner that hangs; on the server, latent requests pile up threads and can drive the instance out of memory. Whether it OOMs or fails its health check, it stops taking traffic, pushing its share onto the remaining instances — which is how one overloaded node becomes a cascading failure, often faster than autoscaling can respond and regardless of autoscaling if no capacity exists.

Fedorka names the terminal state congestive failure: utilization pinned at 100% with everything failing. Two properties make it uniquely bad. The transition from healthy to fully failing is abrupt, leaving no window of graceful degradation. And it does not self-recover — when load returns to a servable level the cluster still serves nothing, and restarting the node costs the restart. The design goal is therefore not "handle the spike" but "never reach this state."

The Buffer Vocabulary

Fedorka's most portable contribution is a vocabulary, and he notes it applies to any shedding implementation, not just Netflix's. In his highway analogy there is usually room for all the cars and even a small spike; that headroom is the success buffer, requests above baseline the cluster can serve successfully. Anyone who has run a cluster deliberately cold has created one. Put a stoplight on the on-ramp, admitting cars only when capacity exists, and you get the failure buffer: capacity reserved for rejecting requests gracefully. Rejecting is far cheaper than serving, so a small reservation buys a large rejection capacity, and you know which requests you rejected, so you can respond deliberately rather than time out.

His example cluster at a 100 RPS baseline serves an extra 30 RPS and gracefully rejects a further 120 RPS, absorbing a 150 RPS spike with no impact to successful requests. Beyond 250 RPS total, successful requests slowly drop as the system trades success buffer to stay functional, returning to normal once load subsides. Because buffer is relative to baseline, halving the baseline on the same cluster changes the picture dramatically.

This yields a workflow: set a buffer goal, work backwards from maximum sustainable RPS, and derive the baseline traffic each instance should take. Fedorka is candid that this is the same practice as running a cluster colder — the contribution is arithmetic that makes it a target you can state and test. You can also shed earlier, shrinking the success buffer to greatly expand the failure buffer. Because the measures are objective, a team can examine a real spike afterwards and say whether the cluster behaved as designed.

Not All Requests Are Equal

Shedding trades a hanging spinner for an immediate error, and Mendiratta answers honestly whether that helps the viewer: it clearly helps the backend engineer, because there is no cascading failure and the error is retryable, so a retry may succeed and the user may never see it. Helping the customer requires a further insight — requests differ in value. Netflix playback has prefetch requests, issued optimistically while you browse (for example on hover), whose failure is invisible beyond slightly higher latency if you later press play; and user-initiated requests, where failure is directly visible. The decisive number: roughly 50% of playback requests were prefetch. Half the load was, in a real sense, optional.

Under equal-opportunity shedding both classes lose availability equally. Under prioritized load shedding the lower priorities go first: in the same experiment prefetch availability dropped sharply while user-initiated stayed at 100%. Mendiratta does not oversell this — push traffic high enough and critical availability falls too. The guarantee is relative: higher priority classes retain higher availability than lower ones.

Two production examples ground the pattern. An infrastructure outage broke playback across all devices; Android clients queued the failing requests to retry later, so recovery brought a huge backlog of prefetch requests at once. Prioritized shedding absorbed it while critical requests stayed available, and Mendiratta notes it likely prevented the backlog from delaying recovery. He volunteers that client-side queueing of non-critical requests is probably poor client design, but backend engineers must handle clients that misbehave. Separately, a popular live event produced a spike of critical traffic, which cannot simply be dropped; critical requests expanded into capacity previously used by lower priorities, so successful critical RPS rose while lower-priority successful RPS fell. Not all critical requests were served, but availability was correctly ordered: critical above degraded above best-effort.

The technique generalizes. Netflix's personalized home page prioritizes foreground requests (you are actively browsing) over background ones (the TV is on but untouched). Data Gateway, the service fronting Netflix's databases, prioritizes writes over reads for two reasons: failed writes cause data loss while reads are retryable, and read RPS is typically much higher, so shedding reads yields more relief per unit of harm.

Running separate clusters for critical and non-critical traffic would also provide isolation, but prioritized shedding provides that isolation at the application level and adds capacity stealing. Because one instance serves both classes, the partition between them is dynamic: during a critical spike, critical requests can expand to consume the whole instance. In a sharded architecture that capacity sits in the other cluster, unreachable without re-provisioning.

Where the Decision Lives

Netflix's first implementation shed at the API gateway using request priority, gateway utilization, and the downstream service's error rate. It worked when the gateway itself was overloaded but had three gaps: no visibility into service utilization, only error rate, which is a lagging indicator appearing after things have gone wrong; no help for backend-to-backend traffic, which never passes the edge, though much of Netflix's low-priority traffic comes from internal batch processes; and no capacity stealing at the service.

The current architecture adds shedding at each service. Service utilization is a leading indicator, so shedding can start before errors appear. It covers backend-to-backend cases — Mendiratta's example is the Viewing History Service, used both for the real-time continue-watching row and by lower-priority batch recommendation jobs. It enables capacity stealing, and because everything is inside one service, teams can run and test it locally. The initial implementation was a Java library; the team is moving it into their Envoy sidecar proxy.

Determining and Communicating Priority

Shedding is a function from (priority, utilization) to a shed/don't-shed decision, and both inputs must be obtained cheaply — you should not burn CPU classifying a request you are about to discard. For priority, Netflix found request headers best, because if the answer is "shed" you never pay to parse the body. The body is more flexible but costs parsing, and a remote lookup is expensive because you make a network round trip before deciding whether to do any work at all. Mendiratta flags the security consequence directly: a header is self-advertised, so a malicious actor can claim high priority, and shedding must be complemented by per-user rate limiting. It is a first line of defence, not the only one.

Priority is assigned at the edge by Netflix's Zuul-based API gateway, whose rules engine considers which API is called and the state of the invoking device, then follows the request through the whole call graph. Services may downgrade priority — making a non-critical call to enrich a critical response is legitimate — but may not upgrade it; an upgrade attempt signals the edge classification is wrong and should be fixed there. Traffic that never touches the edge, such as batch jobs, receives an initial priority through the same mechanism.

Propagation uses self-propagating headers atop Contextflow, Netflix's general request-metadata mechanism, also used for identity, failure injection, and canary routing overrides. A server reads the priority header into lookaside local storage — in Java, a ThreadLocal — which decouples the wire format from every business implementation, and any outgoing call made to complete that work item re-reads that storage and sets the same header, so priority stays attached as the call tree fans out. Asked by his track lead what was actually in the header, Fedorka showed netflix-contextflow-mesh-bin: CAE=: Base64-encoded Protobuf binary decoding to something as simple as requestPolicy priority 1.

Measuring Utilization

CPU is the natural signal for "this service is overloaded," but it is insufficient, because a slow downstream service or database may not raise your CPU at all. For that Netflix uses latency or concurrency, which spike in a highly correlated way. Mendiratta describes the latency-based shedder as a smarter timeout: a plain timeout is a step function, whereas latency-driven prioritized shedding degrades progressively and protects what matters most.

Utilization is normalized so signals are comparable. For latency it is the percentage of requests exceeding the SLO: with a 1-second SLO you might set non-critical shedding at 10% and critical at 40%. Mendiratta concedes plainly that this must be tuned per service and that doing so is tedious — exactly the problem the second half of the talk solves. CPU thresholds are set relative to the autoscaling target, and ordering matters: if autoscaling targets 50% CPU, non-critical shedding might start at 60% and critical at 80%, so the system is already adding capacity while it sheds. In one experiment, once traffic passed 60%, CPU growth slowed even as traffic ramped at the same rate, because rejecting is cheaper than serving; the reported result was handling up to six times the successful RPS without increasing latency for the requests served.

The unit of work being shed differs by service style, and the team targeted their most common web service types.

GraphQL is hardest: even with registered queries, different invocations of the same query can have very different performance profiles, so the team instruments field resolvers. GraphQL also supports field-level errors and partial responses, which Netflix deliberately does not use for shedding. Fedorka's reasoning is worth internalizing: the server does not know whether the caller could use a partial response and will not spend constrained resources producing something unusable, so if a request is low-priority enough that partial data would do, it is low-priority enough to shed entirely. Asked whether this applies to queue consumers, he said he believes yes — decide as you select items for processing, and instrument the underlying business logic — but was explicit that Netflix has not implemented it. That is a hypothesis, not a reported result.

Validating That Priorities Are Correct

Netflix had some priorities before this effort — writes above reads, member device traffic above batch — but Fedorka is direct that it was a weak start. Many were missing, most had never been validated, and teams classifying their own requests got some wrong in both directions: some "important" requests turned out to have adequate fallbacks, while others were inadvertently critical.

The fix is an experiment rather than an opinion. Simplify to two priorities: a request is high priority if failing it moves a metric the business cares about, such as whether a member successfully played their next episode. Then use failure injection testing — annotating a request so it is forced to fail at a chosen point in the call graph — to test the hypothesis: failing a high-priority request should move the metric, failing a low-priority one should not. Fedorka addresses the obvious objection head-on: "my goal is to entertain the world, so I don't really want to stop a member from watching an episode." The resolution is that if they can record that playback would have failed, they can transparently retry without the injected failure, capturing the signal without member harm. This matters most precisely because critical requests may sometimes have to be shed, so you need confidence the labels are right.

Selection is its own problem, since Netflix has thousands of clusters rather than hundreds. Clusters that already caused an incident are easy picks, but rather than wait for incidents, ChAP — Netflix's chaos automation platform — uses failure injection to actively measure the impact of a cluster failing a request. Tracing which priorities flow into each service identifies unprotected clusters receiving high-priority traffic. Finally, application owners confirm the business-domain impact of their cluster going unavailable, closing blind spots and letting the platform reason about each cluster's risk to specific parts of the business. That assessment sets how much buffer each cluster must reserve, verified by routinely passing load tests.

Architecture And Data Flow

Utilization providers write ORCA messages — Open Request Cost Aggregation, an open-source message format — into a directory shared by all processes on the instance. Providers include the application's own RPC latency instrumentation and external contributors such as the data platform team publishing utilization for connected datastores. This is fully out-of-band: no individual request carries load information. The Envoy-based ingress proxy watches the directory with inotify, a Linux kernel feature that efficiently reports file activity in a directory, so updates are picked up without polling.

flowchart TD
    Edge["Zuul API gateway
rules engine assigns priority"] -->|"contextflow header"| Proxy
    Batch["Batch / backend caller
initial priority applied"] -->|"contextflow header"| Proxy
    subgraph Instance["Service instance"]
        Proxy["Envoy sidecar proxy"]
        App["Application"]
        Orca[("ORCA files
shared directory")]
    end
    Proxy -->|"read utilization via inotify"| Orca
    Proxy -->|"shed: graceful reject,
app never sees it"| Reject["Rejected response"]
    Proxy -->|"serve"| App
    App -->|"observed latency"| Orca
    App -->|"downstream call,
priority propagated"| Proxy
    Proxy --> Next["Next service
makes its own decision"]
    Proxy -.->|"metrics and traces"| Obs[("Observability")]
    App -.-> Obs

A call arrives at the sidecar; the proxy inspects request metadata and current ORCA utilization and decides. A shed request is rejected gracefully and never reaches the application; a served request is forwarded, and its execution latency feeds back into the latency utilization written to ORCA. Downstream calls leave through the same proxy, and the receiving service makes its own independent decision. Every interaction is instrumented into metrics and tracing databases, so behaviour is fully observable to teams and tooling.

The decision itself is probabilistic rather than a hard switch. For each utilization signal and cluster, Netflix defines a function mapping (utilization value, priority) to a probability of shedding. At low utilization that probability is zero for every priority; as utilization rises, the probability of shedding low-priority requests grows incrementally, arranged so a lower priority reaches 100% before the next priority up is at any risk; at sufficiently high utilization everything is shed with certainty, keeping the server functional and enabling fast recovery. Each cluster receives a different priority mix, so each gets its own function, designed to produce a smooth throttling response.

Generating and Shipping the Configuration

Per-cluster shedding functions, per-RPC latency distributions, and per-cluster priority mixes add up to a large number of tunables. Fedorka would rather not have them: the team spent years trying to track per-service latency targets manually, and shifting call patterns during load spikes caused the system to both over-respond and under-respond. Beyond correctness, he frames it as platform philosophy — imposing that burden on hundreds of teams is not the experience he wants to provide.

So the platform generates it. A system integrates historical RPC latency, RPC volumes, request priorities, and CPU data to produce a resilience configuration per cluster: expected ranges for each utilization signal, the shedding response function for each utilization and priority, and the details needed to tune utilization monitoring. It regenerates on demand as services change. Early in the rollout, including through several large content events, overrides arrived as Slack messages to a support channel, tracked in a JSON file and merged with the recommendation engine's output; the team deliberately deferred a UI until the system proved its value, and today there is one for viewing and proposing changes. Load shedding and autoscaling share a single configuration because they need the same information. Fedorka frames the relationship carefully: the talk is predicated on not relying on autoscaling, but Netflix clusters do scale continuously with load, so in the common case shedding is a bridge covering the few minutes a scale-up takes. It earns its keep in the other case — when there is no capacity to scale into, or new instances will not become healthy — where it can protect critical traffic for an extended period.

Fedorka admits the discomfort: "I just described a robot that's going to reconfigure how our clusters reject traffic." His answer is that it is about as scary as letting humans do it, which they also allow, and both are made safe by the same objective test. Every generated configuration goes through a ChAP experiment before promotion. ChAP creates test and baseline clusters, routes traffic to warm them to baseline, then rapidly applies a load spike to the canary, and judges automatically: did the cluster receive the expected spike, did it demonstrate the expected success and failure buffers, was all shed load rejected gracefully, and did the RPCs it chose to serve stay within SLO? A pass is promoted; a failure is flagged for human review and parameter adjustment. Throughout, ChAP watches key metrics for member impact and automatically aborts all running tests on any detected impact, even when the cause cannot be attributed. In Fedorka's clean-pass example, clusters were warmed to baseline, a 4x total (3x incremental) spike hit the canary, the cluster briefly shed load, three servers were added, and within roughly three minutes it served all calls.

Scheduling is a constraint problem Fedorka compares to an algorithms homework question: at most one test per region at a time to isolate problems; enough traffic routed to the cluster to make the experiment meaningful, which for some clusters is only a few hours a day and at different hours in different regions; and preferably within the on-call engineer's working hours.

Promotion is deliberately unexciting. The passing configuration becomes an automated pull request against the application's own codebase and rides the normal pipeline — test, canary, gradual rollout — with no special rollback path. The pull request makes the change visible to owners even if it auto-merges, because it appears in a diff between deployed versions, and the configuration is attached to a specific build. Fedorka is unusually forthcoming that this was his second attempt: he originally wanted a fully managed, zero-cognitive-load experience, so he packaged the configuration as a dependency delivered through automated dependency updates, with files inside a ZIP artifact on the application image and a version that always incremented even when nothing changed. Visibility suffered — changes were hard to observe, so it was hard to tell whether a configuration change was associated with a bad build, and viewing the actual configuration meant taking a version from an artifact and leaving your codebase for another application. Moving it into a plain file in the application repository made it easy to find and understand. The generalizable lesson is that "managed" is not the same as "invisible."

None of this runs once; Netflix runs a continuous campaign verifying systems operate with their expected buffer, and ahead of a high-profile content release any system falling short is flagged for review and risk mitigation. Fedorka gives one warning emphatically: do not run automated load tests against production backends without first warning your on-call engineers. Teams have alerts for degradation, and a spike in load shedding is exactly what you want them to investigate normally; even if canaries are allowed some deviation, they talk to real production clusters, which will notice. Netflix leans on SlackOps to notify owners when tests are scheduled, start, and end, and gives teams a large abort button.

Retry Storms and Prioritized Attempt Budgets

Efficient rejection creates a second-order problem, because clients retry — Fedorka's own team builds the IPC clients that do so. A client calls an overloaded cluster, the cluster sheds, the client retries, the cluster is still overloaded, and it sheds again. Without backoff and with retries at multiple layers this grows exponentially; even with correct single-layer retries you can double the traffic to a cluster failing because of overload. Rejection consumes already-constrained resources, and if the retry also fails those resources bought nothing — the same backoff response could have been served earlier and faster.

The simple fix is prioritized backoff: when a low-priority call is shed, do not retry it; fail the whole chain or serve a degraded fallback. Fedorka calls this "really not ideal," because if a single node went unhealthy the retry could have been routed to a healthy node and succeeded, which beats a fallback. The refinement is a prioritized attempt budget: across all calls from one client node to a target cluster, compute the ratio of retries to initial attempts; a low ratio suggests one bad node that will be replaced shortly, so allow all retries, and as the ratio rises, progressively throttle. That much is client-side throttling as described in the SRE book. Netflix's extension is integrating priority: because priority determination is coordinated it is also available to clients, so a client detecting overload progressively stops retrying low-priority calls first, letting high-priority calls retry while low-priority ones sacrifice their share; if the server is shedding aggressively, no calls are retried, since they are unlikely to succeed. This deliberately mirrors server-side shedding — it is load shedding on the client side. Notably it needs no live updates from a central control plane, running entirely from configuration available at application startup, and like load shedding it lives in the service mesh proxy.

Trade-offs And Limitations

• Priority guarantees are relative, not absolute. Mendiratta states plainly that critical availability will eventually drop if traffic keeps climbing, as the live-event example showed. Prioritization guarantees ordering, not a floor.
• Header-based priority is trivially forgeable. It is chosen because it is cheap, and cheapness is the whole point when deciding whether to spend resources on a request. The stated mitigation is per-user rate limiting; anyone adopting this on an untrusted edge needs an equivalent.
• Half of Netflix's playback load was optional. That 50% prefetch share is what makes prioritized shedding so effective here. A service whose traffic is uniformly critical still avoids congestive failure, but has no low-value pool to sacrifice.
• Latency thresholds must be tuned per service, which Mendiratta calls tedious. Netflix's answer is config generation plus chaos validation, a substantial investment. Added context: a smaller organization can adopt the buffer vocabulary and per-service shedding without building ChAP, but should expect the manual tuning burden Netflix spent years avoiding.
• Automated load testing in production has real blast radius, hence the one-test-per-region rule, the traffic and working-hours constraints, the notifications, the abort button, and the automatic global stop on impact.
• Request cost is assumed uniform. Asked in the Q&A whether cost is modelled from headers or payload, Fedorka said utilizations are designed to be independent of request complexity — visible in instrumenting GraphQL data fetchers rather than top-level requests — and that all requests are currently assumed to have a single cost. Differing profiles are handled by tracking multiple points on the latency distribution (for example 60% of calls under one target and 90% under another) rather than communicating per-request cost. He emphasized one requirement: utilization must be monotonically increasing for the shedding function to behave well.
• Queue-based workloads are untested, and GraphQL partial responses are deliberately unused for degradation; a service that knows its callers can use partial data might decide differently.
• Shedding is a bridge, not a substitute for capacity. The common case is still that shedding buys the minutes autoscaling needs; its unique value appears when capacity is genuinely unavailable.

Practical Takeaways

1. Measure your buffers before changing anything. Fedorka says explicitly that if your organization is not ready for any of these implementations, you are at least ready to quantify each cluster's success and failure buffer. Ramp load against a canary and record where successful RPS stops growing and where it starts falling.
2. Set a buffer goal and derive the baseline from it, working back from maximum sustainable RPS to the baseline traffic each instance should take.
3. Find your prefetch equivalent. Audit what fraction of traffic is optional: speculative fetches, background refreshes, batch and analytics jobs, internal reporting. That fraction bounds how much prioritized shedding can help.
4. Validate priority labels experimentally, not by survey. Fail a class of request deliberately, check whether a business metric moves, and retry transparently so real users are not harmed.
5. Assign priority at the edge and propagate it via a cheap carrier such as a header plus lookaside thread-local storage. Allow downgrades, forbid upgrades, and treat an upgrade attempt as a bug in edge classification.
6. Shed at the service, not only at the gateway. Service utilization leads the error rate, and only service-level shedding covers backend-to-backend and batch traffic or enables capacity stealing.
7. Order thresholds relative to autoscaling — scale-up first, then non-critical shedding, then critical shedding.
8. Use a probability curve rather than a hard cutoff, arranged so each priority reaches 100% shed before the next is touched, and confirm your utilization signal is monotonic.
9. Fix retries at the same time as shedding. Suppress retries for low-priority calls first, and use a client-side ratio of retries to initial attempts to distinguish one bad node from a genuinely overloaded cluster.
10. Ship generated configuration through your normal deployment pipeline, as a visible file in the application repository, so it canaries, rolls out, and rolls back like any other change.
11. Warn on-call engineers before automated load tests, provide an abort control, and wire an automatic global stop on any detected user impact.

Key Terms

• Load shedding — Rejecting requests when total demand exceeds provisioned capacity, to preserve latency and availability for what is served. Distinct from rate limiting, which enforces a fixed per-user limit.
• Congestive failure — The terminal overload state where utilization is pinned at 100%, all requests fail, and the system does not recover even after load returns to a servable level.
• Success buffer — Traffic above baseline a cluster can serve successfully, expressed as a multiple of baseline.
• Failure buffer — Traffic above the success buffer that a cluster can reject gracefully without harming successful requests, also a multiple of baseline.
• Prioritized load shedding — Shedding in ascending order of priority so lower-value traffic is dropped before higher-value traffic.
• Capacity stealing — The property, unique to shared-instance prioritized shedding, that high-priority traffic can expand into capacity normally used by low-priority traffic.
• Prefetch request — A speculative client-side fetch whose failure is invisible to the user beyond slightly higher latency later.
• Thundering herd — A large, correlated burst of simultaneous requests, such as many users refreshing at once after a shared failure.
• ORCA (Open Request Cost Aggregation) — An open-source message format used here to publish per-instance utilization out-of-band via a shared directory.
• inotify — A Linux kernel facility for efficiently monitoring file activity in a directory, used to pick up ORCA updates without polling.
• Contextflow — Netflix's mechanism for attaching metadata to a request and propagating it to subsequent calls; carries identity, failure injection, canary routing overrides, and request priority.
• Zuul — Netflix's open-source API gateway, running the rules engine that assigns initial request priority at the edge.
• ChAP — Netflix's chaos automation platform, which runs failure injection tests and the automated load experiments validating resilience configurations.
• Failure injection testing — Annotating a request so it is forced to fail at a chosen point in the call graph, used to find valuable clusters and validate priority labels.
• Resilience configuration — The generated per-cluster artifact holding expected utilization ranges, shedding response functions per utilization and priority, and utilization monitoring details.
• Prioritized backoff — Suppressing retries entirely for load-shed low-priority calls, falling back to a degraded response.
• Prioritized attempt budget — Client-side throttling based on the ratio of retries to initial attempts, extended with priority so low-priority retries are throttled before high-priority ones.

The engineering content here is a sidecar dropping requests along a probability curve, but the more transferable idea is the one Fedorka puts first in his recap: they made overload measurable. Success buffer and failure buffer turn "the cluster fell over" into a number you can set a goal for, generate a configuration from, and test in an experiment that either passes or fails. Every automation in the second half of the talk depends on that objective test existing. If you take one thing from these notes, take the buffer vocabulary — it costs nothing to adopt and everything else is built on it.

Reference: Anirudh Mendiratta and Benjamin Fedorka, Enhancing Reliability Using Service-Level Prioritized Load Shedding at Netflix, QCon San Francisco 2025, published by InfoQ on July 2, 2026.