D1 · Agentic Architecture › Cluster 6 · Reliability

6.7 · Timeouts (Per-Step & Chain)

Bound every hop — latency in a chain accumulates

CONCEPT Three levels of timeout

Timeouts turn an unbounded wait into a controllable failure you can retry, fall back on, or escalate. In an agent there are three distinct levels, and confusing them is a common design bug.

LevelBoundsHow to set
SDK request timeoutOne messages.create call to the Anthropic APIAnthropic(timeout=…) or client.with_options(timeout=…) — default 10 minutes
Per-tool timeoutOne tool invocation (a DB query, an HTTP call)Your own timeout on the tool call — don't let a slow tool hang the loop
Chain / wall-clock budgetThe whole multi-step taskTrack elapsed time across steps; abort when the budget is spent
🔑 The SDK's default request timeout is 10 minutes. That's per API call — a chain of sequential calls can far exceed it in total. A per-call timeout does not bound the whole chain.
⚠️ Raw httpx/requests timeouts are per-read, not wall-clock: a response that keeps trickling bytes can exceed your intended limit because each individual read resets the clock. The SDK's timeout is the right knob for API calls; for a true wall-clock cap on a chain, track elapsed time yourself.

DIAGRAM Chain latency accumulates

flowchart LR S([Request · budget 30s]) --> A["Step 1
timeout 10s"] A --> B["Step 2
timeout 10s"] B --> C["Step 3
timeout 10s"] C --> D{"Σ ≤ budget?"} D -->|Yes| OK["🟢 Return"] D -->|No| TO["🔴 Timeout chain
→ parallelise independents
or abort on budget"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class TO bad;

Three 10-second steps run serially can take 30 seconds even though each is individually "fast." That's the timeout chain failure mode from 6.1 — the fix is to parallelise independent steps and set a per-step deadline, plus an overall budget.

CODE SDK timeout + per-tool timeout + chain budget

import time, anthropic

# SDK request timeout: override the 10-minute default for THIS call.
client = anthropic.Anthropic(timeout=60.0)     # client-wide, seconds

resp = client.with_options(timeout=20.0).messages.create(
    model="claude-sonnet-4-6", max_tokens=1024,   # per-request override
    messages=[{"role": "user", "content": prompt}],
)

# Per-tool timeout: never let one slow tool hang the whole agent loop.
def run_tool(name, args, deadline=5.0):
    try:
        return TOOLS[name](**args, timeout=deadline)
    except TimeoutError:
        # Signal failure so the model adapts (6.4), don't hang.
        return {"content": f"{name} timed out after {deadline}s", "is_error": True}

# Chain budget: bound the WHOLE task, not just each hop.
budget, start = 30.0, time.monotonic()
while not_done():
    if time.monotonic() - start > budget:        # wall-clock cap you track yourself
        escalate("chain exceeded 30s budget")      # abort → escalate (6.5)
        break
    step()
Set the SDK timeout per call, give each tool its own deadline, and track a wall-clock budget for the chain. Independent steps? Parallelise them (6.1) so their timeouts overlap instead of stacking.

TRADE-OFFS Tight vs. generous timeouts

✅ Tight timeouts

  • Fail fast; free resources; snappy UX
  • Surface slow dependencies early (feeds the breaker, 6.2)

⚠️ Too tight

  • Kill legitimately slow work (large model responses, big queries)
  • Trigger needless retries → more load, possible 429s
  • Retrying non-idempotent work risks double-execute (6.6)

EXAM TRAPS

Trap: "The SDK default timeout of 10 min protects the whole chain." → It's per API call. A chain of calls can exceed it in aggregate; track a separate wall-clock budget.
Trap: Relying on raw httpx/requests timeouts as a wall-clock cap. → They're per-read; a slowly-streaming response can blow past your intended limit.
Trap: No per-tool timeout. → One hung tool stalls the entire agent loop. Give each tool its own deadline and signal is_error on timeout so the model adapts.
Trap: Running independent steps serially "to be safe." → That's the timeout chain. Parallelise independents so their latencies overlap rather than sum.

PRACTICE

Q1 · How do you override the Anthropic SDK's request timeout for a single call?

client.with_options(timeout=…).messages.create(...) for one call, or Anthropic(timeout=…) to change it client-wide. The default is 10 minutes per request.

Q2 · Why isn't a raw requests/httpx timeout a reliable wall-clock limit?

Those timeouts are per-read, not total elapsed. A response that keeps sending small chunks resets the read clock each time and can run far longer than you intended. Track elapsed time yourself for a true wall-clock cap.

Q3 · Four independent enrichment lookups each take ~5s and your budget is 8s. What's the fix?

Parallelise them — since they're independent, run concurrently so total ≈ 5s (the slowest), not 20s. Add a per-lookup timeout so one slow lookup can't blow the 8s budget.