Circuit Breaker Pattern: States, Thresholds, Retries and Fallbacks

A payment provider starts answering in ten seconds instead of fifty milliseconds. Your checkout service keeps calling it, every request holds a worker while it waits, and within a minute checkout has no free workers left, including for requests that never needed the payment provider. Then the services that call checkout start timing out too. One slow dependency has become an outage across several teams.

A circuit breaker stops this chain by refusing to call a dependency that is clearly failing, for a while, and answering immediately instead. This post covers the three states with a state diagram, the thresholds that decide when the breaker trips, how it combines with timeouts, retries and bulkheads, what makes a good fallback, and where to put the breaker: in your code or in a service mesh.

Why failures cascade

The arithmetic is simple and unforgiving. A service with 200 worker threads calling a dependency that answers in 50 ms can handle about 4,000 requests per second through that path. If the dependency slows to a 10-second timeout, the same 200 threads handle 20 requests per second. Nothing is broken in your service; its capacity has simply been consumed by waiting.

Retries make it worse. If three layers of services each retry three times, one user request can turn into 27 calls to the struggling dependency at the bottom, at exactly the moment it can least afford them.

Michael Nygard described the circuit breaker in his 2007 book on building software for production, and Martin Fowler's description made it a standard part of service design: wrap calls to a dependency, count failures, and when they cross a threshold, stop calling for a while.

The three states

The open state does two jobs at once. It protects your service, because failing fast frees workers instantly. It also protects the dependency, because it gets a break from traffic while it recovers, instead of a growing pile of retries.

Half-open exists because the breaker cannot know when the dependency is healthy again without trying. Letting through a handful of calls, not the full traffic, tests recovery without knocking the dependency over the moment it comes back.

What counts as a failure

Decide this deliberately, because the wrong choice opens breakers for the wrong reasons:

• Count: timeouts, connection errors, 5xx responses and calls slower than your slow-call threshold.
• Do not count: 4xx responses such as 400 or 404. They describe a bad request, not a failing dependency, and one misbehaving client should not cut everyone off.
• Decide case by case: 429 Too Many Requests. It means the dependency is protecting itself, which is often a good reason to back off.

Slow calls matter as much as errors. A dependency that answers every request successfully in eight seconds is, for an interactive API, a failing dependency.

Choosing thresholds

Resilience4j, a widely used breaker library on the JVM, ships with conservative defaults that are worth knowing because many teams never change them:

For an interactive API, the slow-call defaults are effectively off: nobody waits 60 seconds for a response. Reasonable starting points are a slow-call threshold near your latency objective, a slow-call rate of 50% to 80%, and an open wait of 10 to 30 seconds. Keep minimumNumberOfCalls meaningful: on a low-traffic endpoint, two failures out of three calls is noise, not an outage. Time-based windows behave better than count-based ones when traffic varies a lot through the day.

Timeouts, retries and bulkheads

A circuit breaker is one of four tools that only work well together:

• Timeouts come first. Without a timeout, a slow call never fails, so the breaker never sees anything to count. Set each timeout below the caller's own timeout, and pass deadlines along the call chain so nobody keeps working on a request that has already been abandoned.
• Retries need limits. Retry only idempotent operations, use exponential backoff with jitter as described in the AWS Architecture Blog, retry at one layer rather than every layer, and cap retries as a share of traffic. Envoy's retry budgets default to 20% of active requests when you enable them.
• Never retry an open breaker. Resilience4j applies retry outside the breaker by default, so each attempt passes through it; configure retries to stop on the "call not permitted" error instead of hammering a breaker that has already decided.
• Bulkheads isolate. Give each dependency its own connection pool or concurrency limit, so a slow payment provider can consume only its own pool, not every worker in the service.

Fallbacks that make sense

When the breaker is open, something has to answer. The right fallback depends on what the data is for:

A fallback must be cheap and must not depend on the thing that just failed. If the honest answer is "this does not work right now", fail fast with a clear error; that is still far better than a timeout.

Where to put the breaker

Envoy's model is worth understanding because it differs from the textbook one. Its circuit breakers are concurrency limits per upstream cluster, by default 1,024 connections, 1,024 pending requests, 1,024 active requests and 3 concurrent retries. Per-host failure detection is a separate feature, outlier detection, which by default ejects a host after five consecutive 5xx responses for 30 seconds, longer on repeat ejections, and never ejects more than 10% of a cluster. The service mesh architecture diagram shows where these proxies sit.

Many teams use both: the mesh for consistent limits and bad-host ejection, and a library where the application needs a real fallback. Netflix's Hystrix, which popularized the pattern, has been in maintenance mode since 2018; Resilience4j is the usual replacement on the JVM.

Observe and test it

A breaker that opens silently turns an outage into a mystery. Emit every state change as a log event and a metric, alert when a breaker stays open, and show breaker states on the same dashboard as the dependency's latency. Then test it: inject latency and errors into the dependency in a staging environment and watch the breaker open, the fallback answer and the breaker close again. The observability architecture diagram covers the metrics and tracing side.

The circuit breaker diagram

How to read it:

• Three states. Closed is the normal state, open fails fast, half-open tests recovery. The dot marks the starting state.
• Transitions carry their conditions: a 50% failure rate over the last 100 calls opens the breaker, a 60-second wait moves it to half-open, and the outcome of the trial calls closes or reopens it.
• The call path below shows the caller, the payment provider behind the breaker, and the fallback that answers while the breaker is open.
• The Markdown block lists the settings with their default values.

Drawing resilience patterns clearly

• Draw the breaker as a state machine. Prose descriptions blur the difference between open and half-open; a diagram makes it obvious.
• Write the numbers on the transitions. "Opens on failures" invites argument; "opens at 50% of the last 100 calls" invites review.
• Show the fallback path, including what the user sees.
• Put timeouts and retries on the call path diagram, so reviewers can multiply retries across layers.

For more on state machine notation, see the state diagram example. The microservices architecture diagram shows where breakers sit between services.

Build it in Archboard

1. Sign in with Google or GitHub and create a diagram.
2. Add three State nodes from the UML pack: Closed, Open and Half-open. Color them green, red and amber.
3. Add a small Ellipse Shape from the Core pack as the initial state and connect it to Closed.
4. Draw the transitions as labeled edges: the failure threshold, the wait duration, trial success and trial failure. Add a Core Text label next to each state saying what happens to calls there.
5. Below the state machine, add a Core Box for the caller, an External System from the C4 Model pack for the payment provider and another Core Box for the fallback, with a dashed edge for the open-state path.
6. Add a Markdown block from the Core pack with the settings and defaults.

The example uses 8 diagram nodes, within the Free plan.

FAQ

What are the three states of a circuit breaker? Closed, where calls pass and failures are counted; open, where calls fail immediately for a wait period; and half-open, where a few trial calls decide whether to close the breaker again or reopen it.

What is the difference between a circuit breaker and a retry? A retry repeats a failed call in the hope that it succeeds. A circuit breaker stops calls entirely when failures are frequent. Used together, retries should be limited and should stop when the breaker is open.

Should 4xx errors open a circuit breaker? Usually not. A 400 or 404 means the request was wrong, not that the dependency failed. Count timeouts, connection errors, 5xx responses and slow calls instead.

Do I need a circuit breaker if I use a service mesh? A mesh gives you concurrency limits and ejection of bad hosts without code changes. You still need an in-process breaker where you want a real fallback, such as a cached value or a degraded feature.

What is a good wait duration for the open state? Long enough for a struggling dependency to recover, short enough that users do not see a long outage after it does. Ten to thirty seconds is a common starting point for interactive APIs.

Start free — one seat, no credit card.