CONCEPT Streaming isn't just for UX — it's a timeout defense
Set "stream": true and the Messages API returns the response incrementally over
server-sent events (SSE) instead of one big blocking payload. The obvious win is UX (tokens
appear live). The reliability win is bigger: a long non-streaming request holds one idle connection open
for the whole generation, and some networks drop idle connections — so a long answer can fail or hit an HTTP
timeout with nothing to show. Streaming keeps bytes flowing, so the connection never sits idle.
max_tokens or generations over ~10 minutes, use
streaming (or the Batches API). The SDKs even refuse a non-streaming request they
estimate will exceed the ~10-minute timeout — because idle-connection drops make it unreliable.CONCEPT The event sequence, and how to recover from a drop
An SSE stream has a fixed high-level shape. Handle unknown event types gracefully — new ones may be added.
| Event | Fires | Carries |
|---|---|---|
message_start | Once, first | The Message shell (empty content) |
content_block_start | Each block begins | Block type + index |
content_block_delta | Per token/chunk | Incremental text / JSON / thinking delta |
content_block_stop | Each block ends | — |
message_delta | Top-level updates | stop_reason + cumulative usage |
message_stop | Once, last | Stream complete |
ping events appear anywhere (ignore them). And an error
event can arrive mid-stream after the initial 200 — e.g. an overloaded_error (the streaming
analog of a 529). A mid-stream error does not follow normal HTTP error handling; you must catch it on the
event stream and fall back / retry (6.1) from the partial you have..get_final_message() / .finalMessage() gives you the complete Message with
streaming's timeout protection but without hand-handling every event.DIAGRAM The SSE lifecycle (with a mid-stream error)
(empty Message)"] --> CBS["content_block_start (index i)"] CBS --> CBD["content_block_delta ×N
(text / json / thinking)"] CBD --> CBST["content_block_stop"] CBST -->|"more blocks"| CBS CBST -->|"done"| MD["message_delta
(stop_reason + cumulative usage)"] MD --> MST["🟢 message_stop"] CBD -.->|"overloaded_error
mid-stream"| ERR["🔴 error event
→ keep partial, retry/fallback (6.1)"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class MST ok; class ERR bad;
CODE Stream with progress, then recover on error
Deltas render live (progress + no idle connection); a mid-stream error keeps the partial and hands off to the fallback ladder from 6.1.
def stream_answer(client, messages): partial = "" try: with client.messages.stream( model="claude-opus-4-8", max_tokens=64000, # large → streaming a must messages=messages) as stream: for text in stream.text_stream: partial += text render(text) # show progress live final = stream.get_final_message() # accumulated Message if final.stop_reason == "max_tokens": return continue_from(partial, messages) # 6.4 return partial except (anthropic.APIStatusError, # incl. mid-stream overloaded anthropic.APIConnectionError): # dropped connection return answer_with_fallback(messages, seed=partial) # 6.1, reuse partial
TRADE-OFFS Streaming vs. one blocking call
✅ Streaming (SSE)
- No idle connection → survives long generations
- Live progress; partial output on disconnect
- Required for large
max_tokens/ >10-min work
⚠️ Non-streaming
- Simpler: one request, one response object
- But idle connection can be dropped mid-generation
- Mid-stream errors & timeouts surface as a hard failure
.get_final_message() when you only need
the finished Message — you get the timeout protection without event-by-event handling. For huge,
non-latency-sensitive jobs, the Batches API is the poll-based alternative.EXAM TRAPS
max_tokens essentially
requires it.error event (e.g. overloaded_error) can arrive mid-stream. Handle it on the event
stream, keep the partial, retry/fallback (6.1).usage as incremental. → The usage in
message_delta is cumulative, not per-delta. Don't sum it across events.ping events entirely.PRACTICE
Q1 · Why stream a long generation instead of making one blocking call?
A blocking call holds one idle connection open for the whole generation; some networks drop idle
connections, so a long answer can fail or hit an HTTP timeout with nothing returned. Streaming keeps bytes flowing
(no idle drop), shows progress, and leaves you a usable partial on disconnect. For large max_tokens
the SDKs require it.
Q2 · An overloaded_error arrives after you already got a 200. How?
On a streaming response the initial HTTP status is 200, but an error event can be emitted mid-stream (the SSE analog of a 529). It bypasses normal HTTP error handling — catch it on the event stream, keep the tokens you have, and retry or fall back (6.1).
Q3 · Name the streaming event sequence at a high level.
message_start → for each block: content_block_start, one or more
content_block_delta, content_block_stop → one or more message_delta (carries
stop_reason + cumulative usage) → message_stop. ping and error
events may be interspersed.