Design a Circuit Breaker
How to design circuit breakers for interviews: closed/open/half-open states, failure thresholds, bulkheads, timeouts, fallbacks, and where they sit vs retries and rate limits.
A circuit breaker stops calling a sick dependency so your service fails fast instead of waiting on timeouts until its own thread pool dies. It pairs with retries, rate limiters, and API gateways. Interviewers want the state machine and the tuning knobs, not a library name drop.
Scope with the framework: per-dependency breaker, error rate window, open duration, half-open probes, and a fallback path.
Functional requirements
- Wrap outbound calls: allow, short-circuit, or probe.
- Track successes/failures in a rolling window or bucketed counters.
- Open the circuit when failure rate or consecutive failures cross a threshold.
- After a cool-down, enter half-open and allow limited trial calls.
- Expose metrics and a manual force-open / force-close switch.
Non-functional requirements
- Decision latency near zero (in-process check).
- Isolation: one dependency failure must not cascade.
- Tunable under incident without redeploy when possible (feature flags).
- Correctness under concurrency - concurrent probes in half-open must be capped.
Fail fast is the feature
An open circuit returns an error (or fallback) immediately. That looks worse in a dashboard than a 30s hang - until you realize hangs exhaust every worker. Sell the trade-off out loud.
State machine
| State | Behaviour |
|---|---|
| Closed | Calls flow; failures counted |
| Open | Calls rejected / fallback; timer running |
| Half-open | Small number of trial calls; success closes, failure reopens |
- Closed → Open when failures/time-window exceed threshold (and volume is large enough to avoid flapping).
- Open → Half-open when cool-down elapses.
- Half-open → Closed after N successes; → Open on first (or threshold) failure.
Where it lives
- Client library next to the HTTP/gRPC stub (most common).
- Sidecar / mesh (Envoy outlier detection) for polyglot fleets.
- Gateway for coarse protection of upstreams.
Count timeouts and 5xx as failures; do not count most 4xx. Combine with bounded timeouts - a breaker without a timeout still threads-block. Bulkheads (separate pools) keep payment calls from starving search.
Fallbacks
- Cached last-good response (caching).
- Default / degraded mode (recommendations empty list).
- Queue for later (webhooks style) when sync is not required.
- Never invent money movement in a fallback.
Worked example
- Checkout calls inventory; error rate hits 50% over 20 requests → Open.
- For 30s, checkout serves “inventory unavailable” quickly.
- Half-open allows 2 probes; both succeed → Closed; traffic resumes.
Interview summary
Draw closed/open/half-open. Name thresholds, cool-downs, and timeouts. Separate breakers per dependency and mention bulkheads. That is the circuit breaker interview.