D5 · Context & Reliability › Cluster 6 · Reliability: Degradation & Recovery

6.2 · Circuit Breakers

Stop hammering a dead dependency

CONCEPT Stop calling something that's clearly down

Retries and fallbacks (6.1) handle a single failed call. But when a dependency — the Messages API, a tool backend, an MCP server — is sustained-down, retrying every request just piles load onto a struggling service, burns your latency budget waiting for timeouts, and makes recovery slower. A circuit breaker watches the failure rate and, once it crosses a threshold, trips open: subsequent calls fail immediately (or route straight to a fallback) without touching the sick dependency.

🔑 Retry says "try again." A circuit breaker says "stop trying for a while." It's the pattern that keeps a struggling dependency from being pushed all the way over — and gives it room to recover.

CONCEPT Three states: closed → open → half-open

The breaker is a tiny state machine wrapped around the call site.

StateBehaviorTransition out
🟢 Closed (normal)Calls pass through; count failuresFailures cross the threshold → Open
🔴 Open (tripped)Calls fail fast / route to fallback — dependency is not touchedAfter a cooldown timer → Half-open
🟠 Half-open (probing)Let one trial request throughSuccess → Closed (recovered); failure → back to Open
The half-open probe is the recovery mechanism: after the cooldown you send a single test call. If it succeeds, close the breaker and resume normal traffic; if it fails, re-open and wait again — so you never dump full load onto a dependency that's still sick.
⚠️ A breaker complements the SDK's per-call retries — it doesn't replace them. Retries smooth over blips; the breaker reacts to a pattern of failures across many calls.

DIAGRAM The breaker state machine

stateDiagram-v2 [*] --> Closed Closed --> Open: failures ≥ threshold Open --> HalfOpen: cooldown elapsed HalfOpen --> Closed: probe succeeds HalfOpen --> Open: probe fails note right of Closed 🟢 calls pass through count failures end note note right of Open 🔴 fail fast / use fallback (6.1) end note note right of HalfOpen 🟠 one trial request end note

CODE A minimal circuit breaker for a model client

When open, calls short-circuit to the fallback from 6.1 instead of hitting the overloaded dependency.

class CircuitBreaker:
    def __init__(self, threshold=5, cooldown=30):
        self.threshold, self.cooldown = threshold, cooldown
        self.failures, self.opened_at = 0, None

    def call(self, fn, *args):
        if self.opened_at:                                # OPEN
            if time() - self.opened_at < self.cooldown:
                raise CircuitOpen()                     # fail fast → 6.1 fallback
            self.opened_at = None                        # cooldown done → HALF-OPEN probe
        try:
            result = fn(*args)                          # trial (half-open) or normal
            self.failures = 0                          # success → CLOSED, reset
            return result
        except (anthropic.APIStatusError,
                anthropic.APIConnectionError):
            self.failures += 1
            if self.failures >= self.threshold:
                self.opened_at = time()               # trip OPEN
            raise

TRADE-OFFS Breaker vs. retry-forever

✅ Circuit breaker

  • Fails fast instead of waiting on timeouts
  • Sheds load so the dependency can recover
  • Half-open probe auto-recovers without a deploy

⚠️ Retry every request forever

  • Piles load onto an already-struggling service
  • Every caller eats the full timeout → cascading slowness
  • No automatic "is it back yet?" signal
Tune two knobs: the failure threshold (how many failures trip it) and the cooldown (how long to stay open). Too twitchy = false trips; too sluggish = wasted calls into a dead dependency.

EXAM TRAPS

Trap: Confusing a circuit breaker with a retry. → Retry re-attempts one call; the breaker reacts to a pattern of failures and stops calling entirely for a cooldown. They're layered, not interchangeable.
Trap: Forgetting the half-open state. → Without a probe, an open breaker never learns the dependency recovered and stays open forever (or needs a manual reset). Half-open is the auto-recovery path.
Trap: Flooding a half-open breaker. → Half-open lets through one trial request, not full traffic. Dumping all callers on the first sign of life re-overloads the recovering service.
Trap: Breaker with no fallback behind it. → An open breaker fails fast — pair it with a fallback (6.1) or graceful degradation (6.3) so "fail fast" still returns something useful.

PRACTICE

Q1 · What does the breaker do while it's open, and why?

It fails fast — rejecting calls immediately (or routing to a fallback) without touching the sick dependency. This sheds load so the struggling service can recover, and stops callers from blocking on timeouts.

Q2 · How does an open breaker ever close again?

After a cooldown timer it moves to half-open and allows a single trial request. If that probe succeeds, it closes (recovered); if it fails, it re-opens and waits another cooldown. That's the automatic recovery loop.

Q3 · How do breakers, retries, and fallbacks fit together?

Layered: the SDK retries individual 429/5xx blips; a circuit breaker trips when failures form a sustained pattern and stops hammering the dependency; while open it hands off to a fallback (6.1) or degraded mode (6.3) so the user still gets a response.