D1 · Agentic Architecture › Cluster 2 · Stop Reasons

2.3 · max_tokens vs. Context-Window-Exceeded

Two different failures, two different fixes

CONCEPT Output cap ≠ context exhaustion

The single most common exam trap in Cluster 2: confusing max_tokens with model_context_window_exceeded. They sound similar but describe opposite ends of the request — one is the output being cut off, the other is the input+output budget being blown. Applying the wrong fix makes things worse.

max_tokensmodel_context_window_exceeded
What happenedThe output reached the max_tokens cap you set and was truncated mid-generation.The context window (all input + output tokens) was exhausted.
NOTNOT "the conversation is too long."NOT an output-cap issue.
Emitted byAny model, whenever output hits the cap.Some Claude 4.x models.
FixRaise max_tokens, stream, or continue the turn.Compact the history, or split the conversation.
ContentValid but incomplete — safe to use / continue.The turn couldn't complete within budget.
⚠️ Raising max_tokens to "fix" a model_context_window_exceeded is exactly wrong — a bigger output reservation leaves less room for context. Compact or split instead.

DIAGRAM Two branches, two remedies

flowchart TD S([stop_reason]) --> A{max_tokens?} A -->|Yes| B["🟠 OUTPUT truncated at cap
content is valid but incomplete"] B --> B1["Fix: raise max_tokens"] B --> B2["Fix: stream the response"] B --> B3["Fix: continue the turn"] A -->|No| C{model_context_window_exceeded?} C -->|Yes| D["🔴 CONTEXT window exhausted
input + output over budget"] D --> D1["Fix: compact history"] D --> D2["Fix: split the conversation"] C -->|No| E["(other stop_reason — see 2.1)"] classDef out fill:#ffe8cc,stroke:#d9480f,color:#7a2e08; classDef ctx fill:#f8d7da,stroke:#dc3545,color:#721c24; class B,B1,B2,B3 out; class D,D1,D2 ctx;

CODE Branching the two cases correctly

def handle_length_stops(client, messages, resp):
    if resp.stop_reason == "max_tokens":
        # OUTPUT hit the cap → truncated. Content is valid but incomplete.
        # Option A: give the model more room and continue the turn.
        messages.append({"role": "assistant", "content": resp.content})
        return client.messages.create(
            model="claude-opus-4-8",
            max_tokens=8192,        # raised cap (or: stream=True)
            messages=messages,
        )

    if resp.stop_reason == "model_context_window_exceeded":
        # CONTEXT window exhausted → raising max_tokens would make it WORSE.
        messages = compact(messages)     # summarize/drop old turns...
        # ...or split the conversation into a fresh thread.
        return client.messages.create(
            model="claude-opus-4-8", max_tokens=4096,
            messages=messages,
        )
Mental model: max_tokens = "I ran out of room to write the answer." model_context_window_exceeded = "I ran out of room to hold the whole conversation."

TRADE-OFFS Fixing max_tokens: raise vs. stream vs. continue

✅ Raise the cap

  • Simplest; one param change
  • Good when outputs are legitimately long
  • Streaming avoids buffering a huge response

⚠️ Watch the cost side

  • A larger max_tokens reservation eats context budget
  • Continuing the turn adds a round-trip & more tokens
  • If truncation is chronic, the real fix may be a smaller task

EXAM TRAPS

Trap: "max_tokens means the conversation is too long." → No. It means the output was truncated at its cap. Conversation-length exhaustion is model_context_window_exceeded.
Trap: Fixing model_context_window_exceeded by raising max_tokens. → That reserves more output room and leaves less for context — it worsens the problem. Compact or split.
Trap: Discarding the truncated max_tokens output. → The content is valid, just incomplete. You can append it and continue the turn rather than starting over.

PRACTICE

Q1 · A long agent run returns model_context_window_exceeded. What do you do?

Reduce the context: compact the history (summarize or drop old turns) or split the conversation into a fresh thread. Do not raise max_tokens — that reserves more output room and shrinks the context budget further.

Q2 · The model returns stop_reason == max_tokens with a half-finished JSON object. Fix?

The output was truncated. Append the assistant turn and continue the turn with a larger max_tokens (or stream). The partial content is valid — you don't throw it away.

Q3 · In one sentence, what distinguishes the two failures?

max_tokens is the output hitting its cap (truncation); model_context_window_exceeded is the whole context window (input + output) being exhausted.