Breakers, bulkheads and shedding load

Timeouts and retries govern one call. These three patterns govern what happens when a dependency is persistently unwell.

A circuit breaker has three states

Closed to open happens when the failure rate over a window crosses a threshold. Open to half-open happens after a wait. Half-open then goes back to closed if the trials succeed, or straight back to open if they do not.

Half-open exists so recovery does not become a second outage. Without it, the breaker would go from open to fully closed and release the entire held-back load at a dependency that has just come back — which knocks it over again, and now you have a loop. A handful of trial calls answers “is it actually better?” at a cost the dependency can survive.

The state that matters most is open, and the thing to understand is who it protects. It is not there to shield the failing service — that service has its own problems. It is there to stop your threads queueing against it, so the parts of your system that do not depend on it keep answering.

Bulkheads isolate the damage

A bulkhead caps how much of a resource one dependency may consume — typically concurrent calls, sometimes a separate thread pool.

The failure it prevents: one slow dependency taking the whole service down. With a single shared pool of 200 threads and one dependency that starts taking ten seconds, all 200 threads end up waiting on it, and requests that never touch it now time out too. Cap that dependency at 20 and the damage is bounded — those calls fail or wait, and everything else is unaffected.

The name comes from ships: compartments so a hole floods one section rather than the hull.

Shedding beats queueing

When arrivals exceed what the service can process, the queue grows. Two things follow, and both are worse than they look:

Latency grows without bound while throughput stays flat. The service is already at capacity; queueing does not add any. It only adds waiting.

Most of the queue is work nobody wants any more. A request that has waited eight seconds is probably attached to a client that gave up at three. You spend capacity computing answers for callers who are gone — and because those responses are wasted, the queue drains even slower.

Shedding — rejecting immediately with a 429 or 503 when the queue is over a bound — keeps latency for accepted work stable and lets the caller decide: retry with backoff, degrade, or tell the user. A bounded queue with a short timeout is the practical form of this; an unbounded queue is a deferred outage.

Where it fits together

timeout       bounds one call
retry         handles a transient failure, safely
bulkhead      bounds how much this dependency can consume
breaker       stops calling something that is persistently down
shedding      protects the service from its own inbox

None replaces another, and there is one thing they all need: a decision about what a degraded answer looks like. “Fail fast” is only useful if failing means something better than a stack trace — a cached price, yesterday’s total, a page that renders without the recommendations panel. That decision is a product decision, and it is the one most often skipped.

The Resilience4j configuration for all five, and the incident that produced the numbers: Resilience4j Patterns.