CONCEPT max_tokens reserves output space inside the window
Every turn splits into two phases that share one window: the input phase
(system prompt + all history + current message + tool defs) and the output phase (the reply).
max_tokens is the cap on that output phase — it reserves room for the reply. The
reservation has to fit: the output you allow plus the input must live inside the same context window,
and the generated output counts against the window just like the input does.
input_tokens + max_tokens ≤ context window. Reserve too little
and the reply truncates; reserve too much and the history can't fit. The output isn't "free space" outside the
window — it's carved from it.CONCEPT Thinking tokens are output, and count too
With extended thinking, the thinking budget is a subset of max_tokens and is
billed as output tokens — and, like every other output token, it counts toward the context
window. With adaptive thinking (the mode on current models) Claude decides its thinking
allocation dynamically, so output usage varies request to request. When you reserve output room, reserve enough
for both the visible reply and any thinking it needs.
stop_reason: "max_tokens" — the reply
is truncated mid-thought. That is a self-imposed output cap, and is not the same as
running out of context window (that's model_context_window_exceeded, covered in 1.5).DIAGRAM The window is split, not extended
(input_tokens)"] W --> RES["OUTPUT: reply + thinking
(≤ max_tokens)"] INP --> CK{"input + max_tokens
≤ window?"} RES --> CK CK -->|"yes"| OK["🟢 Fits — safe to send"] CK -->|"no"| BAD["🔴 Shrink history or
lower max_tokens"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OK ok; class RES gate; class BAD bad;
CODE Size max_tokens against the remaining window
WINDOW = 1_000_000 # Opus 4.8 — 1M, default SAFETY = 2_000 # leave slack for count drift input_tokens = client.messages.count_tokens( model="claude-opus-4-8", messages=history, ).input_tokens # Reserve output from whatever the input leaves behind. room = WINDOW - input_tokens - SAFETY max_out = min(8_000, room) # cap the reply to what actually fits if max_out <= 0: history = trim_history(history) # no room for a reply → make room (1.4/1.5) resp = client.messages.create( model="claude-opus-4-8", max_tokens=max_out, # output reservation, inside the window thinking={"type": "adaptive"}, # thinking is billed as output, counts too messages=history, ) if resp.stop_reason == "max_tokens": ... # reply hit the reservation — raise max_out or continue
TRADE-OFFS How big should max_tokens be?
| Setting | Upside | Downside |
|---|---|---|
Generous max_tokens | Reply won't truncate; room for thinking | Eats window; less room for history; higher worst-case cost/latency |
Tight max_tokens | More room for history; predictable cost | Risk of stop_reason: "max_tokens" mid-answer |
max_tokens from the remaining window
(window − input − safety), not a fixed number — the input side grows every turn (1.4).EXAM TRAPS
max_tokens is separate from the context window." → No. Output is
reserved inside the window: input + max_tokens must fit, and output counts toward the
window.max_tokens huge to be safe." → A huge reservation crowds
out history and can push input + max_tokens past the window. Size it to what actually fits.max_tokens, billed as output, and counts toward the window.stop_reason: max_tokens means I'm out of context." → It means you
hit your output cap. Out-of-context is a different stop reason,
model_context_window_exceeded (1.5).PRACTICE
Q1 · What does max_tokens control, and where does that budget come from?
It caps the output phase (the reply). The reservation is carved from the same context
window as the input — input_tokens + max_tokens must fit, and generated output counts toward the
window.
Q2 · Do extended-thinking tokens count against max_tokens and the window?
Yes to both. Thinking is a subset of max_tokens, billed as output tokens,
and counts toward the context window. Adaptive thinking sizes it dynamically per request.
Q3 · Why compute max_tokens from the remaining window rather than hard-coding it?
Because the input side grows every turn (1.4). A fixed max_tokens that fit early can
later push input + max_tokens past the window. Deriving it from window − input − safety
keeps the reservation valid as history accumulates.