D5 · Context & Reliability › Cluster 1 · Context Windows & Token Budgeting

1.5 · Context-Window-Exceeded Handling

Detect & recover, don't crash

CONCEPT Two distinct overflow conditions

When a request is too big for the window, the API's behavior depends on which way it's too big:

ConditionWhat the API does
Input alone exceeds the windowRejected up front: HTTP 400 invalid_request_error ("prompt is too long"), on every model.
Input + max_tokens exceeds the window (Claude 4.5+)Request is accepted. If generation then reaches the window limit, it stops with stop_reason: "model_context_window_exceeded".
Same case on earlier modelsReturns a validation error instead — opt into the graceful stop with beta header model-context-window-exceeded-2025-08-26.
⚠️ model_context_window_exceeded is not the same as max_tokens. max_tokens means you hit your own requested output cap (1.3); model_context_window_exceeded means generation ran into the model's hard window limit. Same symptom (a truncated reply), different cause and different fix.

CONCEPT Recover by making room, then continue

These are handleable stop conditions, not crashes. The response you got is valid partial output; the fix is to reduce the input and continue:

Design move: never let a 400 or a context-exceeded stop reach the user as an error. Detect the condition, trim or summarize the history, and re-issue the turn.

DIAGRAM Route each overflow to a recovery

flowchart TD R["Send request"] --> Q{"Outcome?"} Q -->|"400 prompt too long"| PRE["🔴 Input alone too big
→ trim/summarize BEFORE resending"] Q -->|"stop: model_context_window_exceeded"| MID["🟠 Ran into window mid-gen
→ compact, then continue"] Q -->|"stop: max_tokens"| CAP["🟠 Hit YOUR output cap
→ raise max_tokens / continue"] Q -->|"stop: end_turn"| OK["🟢 Normal completion"] PRE --> R MID --> R classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class MID gate; class CAP gate; class PRE bad;

CODE Detect the distinct stop reason and recover

import anthropic

def send_turn(history, max_out=4096):
    try:
        resp = client.messages.create(
            model="claude-opus-4-8", max_tokens=max_out, messages=history,
        )
    except anthropic.BadRequestError:
        # 400 "prompt is too long" — input alone overflowed. Shrink & retry.
        compact(history)
        return send_turn(history, max_out)

    # Context-window-exceeded is a STOP REASON, not an exception (Claude 4.5+).
    if resp.stop_reason == "model_context_window_exceeded":
        compact(history)          # make room, then keep the partial going
        return send_turn(history, max_out)

    if resp.stop_reason == "max_tokens":
        ...                        # DIFFERENT: your own cap — raise it or continue

    return resp   # end_turn / tool_use / etc.
🔑 On earlier models the second case is a validation error unless you send the model-context-window-exceeded-2025-08-26 beta header to get the graceful stop reason.

TRADE-OFFS Prevent vs. react

✅ Prevent (count first)

  • No wasted create call, no user-visible error
  • Free pre-check; own rate-limit pool
  • Deterministic — you decide when to trim

⚠️ React (handle the stop)

  • Simpler loop, no pre-count round trip
  • But: a create call is spent, output truncates
  • Must correctly distinguish the two stop reasons
Best of both: count_tokens to prevent the common case, and handle the stop reason as a safety net. Compaction (beta) automates the recovery for long-running chats.

EXAM TRAPS

Trap: "model_context_window_exceeded and max_tokens are the same thing." → No. One is the model's hard window limit; the other is your own output cap. Different cause, different fix.
Trap: "Context overflow always throws an exception." → When input + max_tokens overflows on Claude 4.5+, the request is accepted and returns a stop reason. Only input-alone-too-big is a 400.
Trap: "Just retry the same request." → Retrying an over-window request repeats the failure. You must reduce the input (trim/summarize/compact) first.
Trap: "The graceful stop works everywhere by default." → On earlier models you need the model-context-window-exceeded-2025-08-26 beta header; otherwise it's a validation error.

PRACTICE

Q1 · How does the API signal "input alone is too big" vs. "ran out of window while generating"?

Input alone over the window → HTTP 400 invalid_request_error ("prompt is too long") on every model. Input + max_tokens over the window (Claude 4.5+) → the request is accepted and generation stops with stop_reason: "model_context_window_exceeded".

Q2 · Why must you not treat model_context_window_exceeded like max_tokens?

They have different causes: max_tokens is the output cap you set (fix: raise it or continue), while model_context_window_exceeded is the model's hard window limit (fix: reduce input via trim/summarize/compact). Bumping max_tokens won't help the latter.

Q3 · What are the two main ways to recover, and which one is automated?

Trim (drop old turns / clear stale tool results) or summarize the history, then continue. On older models you need the model-context-window-exceeded-2025-08-26 beta header for the graceful stop. Server-side compaction (beta) automates summarization so the conversation continues past the window limit.