CONCEPT stop_reason: "max_tokens" means the answer is cut off, not done
Every response carries a stop_reason. When Claude finishes naturally it's end_turn;
when it wants a tool it's tool_use. But stop_reason: "max_tokens" means
the model hit your max_tokens ceiling mid-generation — the text is truncated,
possibly mid-sentence or mid-JSON. The single biggest reliability bug here is treating that partial output as a
complete answer.
stop_reason before consuming content. If it's
max_tokens, the response is incomplete — you either raise the ceiling and retry, or
continue generation from where it stopped.json.loads() throws on
a half-written object. Check stop_reason first; don't parse a max_tokens body as if it
were whole.CONCEPT Continue: feed the partial back, ask it to keep going
To recover a cut-off answer, append the model's partial output as an assistant turn, then add a short user message asking it to continue. The model sees its own prior text as context and picks up where it left off; you concatenate the two halves.
{assistant: partial} then
{user: "Continue from where you left off."} — the assistant block is history, not a prefill.
max_tokens. Too low →
routine truncation and continuation round-trips. Too high without streaming → the SDK refuses / risks an HTTP
timeout on long generations (see 5.5 / 6.5). Size it to the expected output, and stream when it's large.DIAGRAM Detect truncation, then continue
'Continue where you left off'"] ASK --> R2["Re-call → concatenate halves"] R2 --> S classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class DONE ok; class CUT gate; class APP gate; class ASK gate;
CODE Detect max_tokens and continue to completion
Loop until end_turn, stitching each partial together. A capped retry count guards against a runaway
continuation loop.
def complete(client, messages, max_tokens=2048, max_conts=4): full = "" for _ in range(max_conts + 1): resp = client.messages.create( model="claude-opus-4-8", max_tokens=max_tokens, messages=messages) part = next(b.text for b in resp.content if b.type == "text") full += part if resp.stop_reason != "max_tokens": # end_turn → done return full # Truncated: feed the partial back as history, then a user nudge. # NOT a trailing assistant turn (that's a prefill → 400 on 4.x models). messages = messages + [ {"role": "assistant", "content": part}, {"role": "user", "content": "Continue exactly where you left off; do not repeat."}, ] return full # hit continuation cap
TRADE-OFFS Continue vs. just raise max_tokens
✅ Continue from the partial
- Recovers work already generated — no wasted tokens
- Handles outputs longer than any single ceiling
- Keeps
max_tokensmodest per call
⚠️ Just crank max_tokens high
- Simpler — one call, no stitching
- But wastes budget when most answers are short
- Very high values need streaming or hit HTTP timeouts
max_tokens to the typical output; keep continuation as the safety
net for the occasional long answer. Stream (6.5) whenever the ceiling is large.EXAM TRAPS
max_tokens response as complete. → It's truncated.
Always check stop_reason; continue or raise the ceiling before using the content.max_tokens cut-off can slice a structured
output mid-object; json.loads() throws. Detect truncation first, then continue or increase
max_tokens.max_tokens enormous with no streaming. →
Large ceilings without streaming risk idle-connection HTTP timeouts (the SDK even refuses them). Stream, or
continue in modest chunks.PRACTICE
Q1 · A response comes back with stop_reason: "max_tokens". Is it done?
No — it was cut off at the token ceiling and is incomplete (maybe mid-sentence or
mid-JSON). Continue generation (feed the partial back + a user "continue" nudge) or raise max_tokens
and retry. Never consume it as a finished answer.
Q2 · Why not just append the partial as a trailing assistant message and re-call?
A trailing assistant message is a prefill, which returns a 400 on current models (Opus 4.8/4.7/4.6, Sonnet 5/4.6, Fable 5). Keep the assistant partial as history and make the last message a user turn asking it to continue.
Q3 · How do you decide the right max_tokens?
Size it to the expected output length: too low forces frequent truncation and continuation round-trips; too high wastes budget and, without streaming, risks HTTP timeouts. Set a sensible default, keep continuation as the safety net, and stream for large ceilings.