CONCEPT The API is stateless — you resend the whole history
The Messages API has no server-side memory. Each turn, your input phase is all previous conversation history plus the current user message, and previous turns are preserved completely. So the input side grows monotonically: every user message and assistant reply accumulates in the window, and you re-send that entire transcript on the next call. The output of one turn becomes part of the input for the next.
CONCEPT Watch the input side climb toward the window
The danger isn't a single giant message — it's accumulation. Each turn the input grows, the room
left for the reply shrinks (1.3), and eventually input + max_tokens presses against the window.
Two things help you stay ahead of it:
- Track it:
usageon every response reports the input consumed; orcount_tokensbefore sending (1.2). - Context awareness (some models): Sonnet 5, Sonnet 4.6, Sonnet 4.5, and Haiku 4.5 automatically track their remaining budget via injected tags (you never send them). Newer models (Opus 4.7+, Fable 5, Mythos 5) don't get these — give them an explicit budget via task budgets (beta) instead.
DIAGRAM Each turn re-sends everything before it
sys + U1"] --> A1["+ A1"] A1 --> T2["Turn 2 input
sys + U1 + A1 + U2"] T2 --> A2["+ A2"] A2 --> T3["Turn 3 input
sys + U1 + A1 + U2 + A2 + U3"] T3 --> GROW{"Approaching
the window?"} GROW -->|"no"| KEEP["🟢 Keep appending"] GROW -->|"yes"| MANAGE["🟠 Compact / trim
the history (1.5)"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class KEEP ok; class MANAGE gate;
CODE Budget check each turn as history accumulates
WINDOW, MAX_OUT, SAFETY = 1_000_000, 4096, 2_000 history = [] # you own the transcript — the API stores nothing def turn(user_msg): history.append({"role": "user", "content": user_msg}) # Input grows every turn — re-check before each send. n_in = client.messages.count_tokens( model="claude-opus-4-8", messages=history, ).input_tokens if n_in + MAX_OUT + SAFETY > WINDOW: compact(history) # summarize/trim oldest turns (1.5) resp = client.messages.create( model="claude-opus-4-8", max_tokens=MAX_OUT, messages=history, ) # Re-sent history is re-billed: this input was paid for last turn too. history.append({"role": "assistant", "content": resp.content}) return resp
TRADE-OFFS Keep full history vs. manage it
✅ Append everything
- Perfect recall of the whole conversation
- Simplest possible loop
- Cache reads soften the re-billing cost
⚠️ Unbounded growth
- Per-turn cost climbs (re-sent, re-billed)
- Latency and context rot rise with size
- Eventually overflows the window (1.5)
EXAM TRAPS
PRACTICE
Q1 · Why does per-turn cost rise as a conversation grows?
The API is stateless, so each turn you re-send the full history. Those input tokens are re-sent and re-billed every turn — early turns get paid for again and again — so cost climbs with length even when new messages are short.
Q2 · Does prompt caching reduce how much of the context window the history uses?
No. Caching reduces the cost of re-sent tokens (cheap cache reads), but cached tokens still occupy the context window. To reduce window usage you must trim or compact (1.5).
Q3 · How can you tell you're approaching the window mid-conversation?
Read usage on each response, or count_tokens before sending. On
context-aware models (Sonnet 5/4.6/4.5, Haiku 4.5) the model itself tracks remaining budget via injected tags;
on Opus 4.7+/Fable 5/Mythos 5, give it a task budget instead.