D5 · Context & Reliability › Cluster 1 · Context Windows & Token Budgeting

1.3 · Input vs Output Tokens & max_tokens

Reserve room for the reply

CONCEPT max_tokens reserves output space inside the window

Every turn splits into two phases that share one window: the input phase (system prompt + all history + current message + tool defs) and the output phase (the reply). max_tokens is the cap on that output phase — it reserves room for the reply. The reservation has to fit: the output you allow plus the input must live inside the same context window, and the generated output counts against the window just like the input does.

🔑 Rule of thumb: input_tokens + max_tokens ≤ context window. Reserve too little and the reply truncates; reserve too much and the history can't fit. The output isn't "free space" outside the window — it's carved from it.

CONCEPT Thinking tokens are output, and count too

With extended thinking, the thinking budget is a subset of max_tokens and is billed as output tokens — and, like every other output token, it counts toward the context window. With adaptive thinking (the mode on current models) Claude decides its thinking allocation dynamically, so output usage varies request to request. When you reserve output room, reserve enough for both the visible reply and any thinking it needs.

⚠️ Hitting the reservation ends the turn with stop_reason: "max_tokens" — the reply is truncated mid-thought. That is a self-imposed output cap, and is not the same as running out of context window (that's model_context_window_exceeded, covered in 1.5).

DIAGRAM The window is split, not extended

flowchart LR W["Context window"] --> INP["INPUT: system + history + tools
(input_tokens)"] W --> RES["OUTPUT: reply + thinking
(≤ max_tokens)"] INP --> CK{"input + max_tokens
≤ window?"} RES --> CK CK -->|"yes"| OK["🟢 Fits — safe to send"] CK -->|"no"| BAD["🔴 Shrink history or
lower max_tokens"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class RES gate; class BAD bad;

CODE Size max_tokens against the remaining window

WINDOW = 1_000_000          # Opus 4.8 — 1M, default
SAFETY = 2_000              # leave slack for count drift

input_tokens = client.messages.count_tokens(
    model="claude-opus-4-8", messages=history,
).input_tokens

# Reserve output from whatever the input leaves behind.
room = WINDOW - input_tokens - SAFETY
max_out = min(8_000, room)   # cap the reply to what actually fits

if max_out <= 0:
    history = trim_history(history)   # no room for a reply → make room (1.4/1.5)

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=max_out,             # output reservation, inside the window
    thinking={"type": "adaptive"},  # thinking is billed as output, counts too
    messages=history,
)
if resp.stop_reason == "max_tokens":
    ...  # reply hit the reservation — raise max_out or continue

TRADE-OFFS How big should max_tokens be?

SettingUpsideDownside
Generous max_tokensReply won't truncate; room for thinkingEats window; less room for history; higher worst-case cost/latency
Tight max_tokensMore room for history; predictable costRisk of stop_reason: "max_tokens" mid-answer
Practical default: pick max_tokens from the remaining window (window − input − safety), not a fixed number — the input side grows every turn (1.4).

EXAM TRAPS

Trap: "max_tokens is separate from the context window." → No. Output is reserved inside the window: input + max_tokens must fit, and output counts toward the window.
Trap: "Just set max_tokens huge to be safe." → A huge reservation crowds out history and can push input + max_tokens past the window. Size it to what actually fits.
Trap: "Thinking tokens are free / don't count." → Thinking is a subset of max_tokens, billed as output, and counts toward the window.
Trap: "stop_reason: max_tokens means I'm out of context." → It means you hit your output cap. Out-of-context is a different stop reason, model_context_window_exceeded (1.5).

PRACTICE

Q1 · What does max_tokens control, and where does that budget come from?

It caps the output phase (the reply). The reservation is carved from the same context window as the input — input_tokens + max_tokens must fit, and generated output counts toward the window.

Q2 · Do extended-thinking tokens count against max_tokens and the window?

Yes to both. Thinking is a subset of max_tokens, billed as output tokens, and counts toward the context window. Adaptive thinking sizes it dynamically per request.

Q3 · Why compute max_tokens from the remaining window rather than hard-coding it?

Because the input side grows every turn (1.4). A fixed max_tokens that fit early can later push input + max_tokens past the window. Deriving it from window − input − safety keeps the reservation valid as history accumulates.