D5 · Context & Reliability › Cluster 5 · Reliability: Errors & Retries

5.2 · Retryable vs Not & Idempotency

Only retry what's safe

CONCEPT Two questions gate every retry

Before your code re-sends a failed request it must answer two separate questions, and both must be "yes":

  1. Is the error retryable? Transient failures — 408 request timeout, 409 conflict, 429 rate limit, and all 5xx (500, 529) plus raw connection errors — can succeed on a second try. Terminal 4xx (400, 401, 403, 404, 413) cannot; retrying is wasted. Among the headline Messages-API codes, 429 is the only retryable 4xx.
  2. Is the operation safe to repeat? This is idempotency. A read (or a request that never landed) is safe to replay. But a request with a side effect that may have partially succeeded — you got a timeout after the server did the work — could double-charge or duplicate if you blindly resend.
🔑 Retryability is about the error; idempotency is about the operation. A retryable error on a non-idempotent operation is the dangerous case — the one that double-charges.
CodeRetryable?Why
400 / 401 / 403 / 404 / 413🔴 NoYour request is wrong — same bytes fail again
408 request timeout🟢 YesTransient — connection/processing timed out
409 conflict🟢 YesResolve the conflicting state, then retry
429 rate_limit🟢 YesValid request, over the per-minute limit
500 / 529🟢 YesAnthropic-side; the request itself was fine

CONCEPT Idempotency: the double-charge trap

A pure Messages API call is read-like — replaying it just costs you tokens twice, no lasting harm. The trap appears when a tool the model triggered has a side effect: charge_card, send_email, create_order. If your harness times out after the payment processor charged the card but before you saw the response, a naive retry runs the charge a second time. The fix is an idempotency key: attach a stable, unique ID to the side-effecting request so the downstream service dedupes replays — the second attempt with the same key returns the first result instead of charging again. Anthropic's own SDKs auto-retry the transient errors above with backoff, and support idempotency keys so those retries stay safe.

Reversible / read-only ops (Messages calls, GETs, retrieval) → retry freely. Irreversible side effects → gate the retry behind an idempotency key, or don't auto-retry at all.
Auto-retrying a non-idempotent charge_card tool on a 429/timeout with no idempotency key → the customer gets billed twice. Retryable error + unsafe operation = the double-charge.

DIAGRAM Both gates must pass

flowchart TD A["Request failed"] --> Q1{"Retryable error?
408/409/429/5xx"} Q1 -->|"no — 400/401/403/404/413"| STOP["🔴 Surface it
don't retry"] Q1 -->|"yes"| Q2{"Operation idempotent?"} Q2 -->|"yes: read / Messages call"| GO["🟢 Retry with backoff"] Q2 -->|"no: side effect"| KEY{"Idempotency key set?"} KEY -->|"yes"| GO KEY -->|"no"| DANGER["🔴 Don't blind-retry
(may double-charge)"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class GO ok; class KEY gate; class STOP bad; class DANGER bad;

CODE A retry predicate that respects both gates

Let the SDK auto-retry the safe Messages calls; hand-roll retries only for side-effecting tools, and only with an idempotency key.

# The SDK already retries 408/409/429/5xx + connection errors, twice by
# default, honoring retry-after. Tune or disable per-client:
client = anthropic.Anthropic(max_retries=4)   # 0 disables auto-retry

RETRYABLE = {408, 409, 429, 500, 529}

def should_retry(status, tool):
    if status not in RETRYABLE:
        return False                    # terminal 4xx — fixing needed
    if tool.side_effecting and not tool.idempotency_key:
        return False                    # unsafe to replay — could double-charge
    return True

# Side-effecting tool call: stable key makes the replay a no-op downstream
charge_card(amount=4200, idempotency_key=f"order-{order_id}")

TRADE-OFFS Auto-retry everything vs. gate by idempotency

✅ Gate side effects by idempotency

  • Reads & Messages calls retry freely — resilient
  • Charges/emails never duplicate
  • Idempotency key makes even the unsafe ops safe to replay

⚠️ Blanket auto-retry on every op

  • One retry policy, less code
  • But: a post-success timeout replays the side effect
  • Double charges, duplicate orders, repeated emails

EXAM TRAPS

Trap: "If the error is retryable, just retry." → Retryable error is necessary, not sufficient. Also ask whether the operation is idempotent. A retryable 429 on a charge_card with no idempotency key can double-charge.
Trap: Thinking a plain Messages API call is unsafe to retry. → It's read-like — a replay just costs tokens again. The danger is side-effecting tools the model triggers, not the model call itself.
Trap: "The SDK retries for me, so I don't need idempotency keys." → The SDK auto-retries transient errors, but for your own side-effecting calls you still supply the idempotency key so a replay dedupes.
Trap: Treating a timeout as "definitely didn't happen." → A timeout means unknown — the work may have completed before you lost the connection. That's exactly why idempotency keys exist.

PRACTICE

Q1 · A 429 comes back on a send_email tool with no idempotency key. Retry?

Not blindly. 429 is retryable, but send_email is non-idempotent — a replay could send a duplicate. Add an idempotency key (so the mail service dedupes) or don't auto-retry this one.

Q2 · Why is a plain messages.create() safe to retry but charge_card isn't?

The Messages call has no lasting side effect — replaying it just regenerates a response (costing tokens). charge_card mutates external state; a replay after a post-success timeout charges twice. Idempotency is a property of the operation, not the error.

Q3 · What role does an idempotency key play in a retry?

It gives the downstream service a stable ID to dedupe replays: the second request with the same key returns the first result instead of performing the side effect again. It turns an unsafe-to-replay operation into a safe-to-retry one.