D5 · Context & Reliability › Cluster 3 · Managing Long Conversations

3.1 · Conversation State & Statelessness

You own the history

CONCEPT The Messages API is stateless

The Claude Messages API keeps no server-side memory of your conversation. Each call to POST /v1/messages is independent: the model sees only what you put in this request. There is no session ID, no thread the server holds onto. To make a multi-turn chat feel continuous, you resend the entire conversation history every turn — the system prompt, every prior user and assistant message, every tool call and tool result — plus the new user message.

🔑 The mental model: the API is a pure function. response = f(system, tools, messages). Same inputs → the model has the same "memory." If you drop a turn from messages, the model genuinely never saw it.

Everything in the request counts toward the context window (up to 1M tokens on current models like Opus 4.8 / Sonnet 5): the system prompt, every message (including tool results, images, documents), and your tool definitions — plus the output Claude generates this turn. Each response reports what the request consumed in its usage field.

CONCEPT "You own the history" — the architectural consequence

Because the server holds nothing, conversation state is your responsibility. Your application is the source of truth: you append each assistant reply to a list, append the next user message, and send the whole list back. This is liberating (you can edit, branch, replay, or persist history however you like) and it's a burden (nobody trims the transcript for you).

As the conversation advances, tokens accumulate monotonically — previous turns are preserved completely, so each turn is bigger and more expensive than the last. Eventually a long-running conversation approaches the context-window limit. That is exactly why the rest of this cluster exists: truncation (3.2), summarization / compaction (3.3), context editing (3.4), and the memory tool (3.5) are the four strategies you apply to the history you own.

⚠️ If input alone already exceeds the window, the API returns 400 invalid_request_error ("prompt is too long"). On Claude 4.5+ models, if input + max_tokens overflows mid-generation you instead get stop_reason: "model_context_window_exceeded". Manage the transcript before you hit either.

DIAGRAM Statelessness forces a resend loop

flowchart TD A["Your app holds
messages[]"] --> B["Append new
user message"] B --> C["POST /v1/messages
(FULL history each turn)"] C --> D["Server: stateless
response = f(system, tools, messages)"] D --> E["Assistant reply
+ usage.input_tokens"] E --> F["Append reply to
YOUR messages[]"] F -->|"next turn"| B F -.->|"history grows every turn →
eventually hits window limit"| G["Apply 3.2–3.5
to the history you own"] classDef app fill:#d4edda,stroke:#28a745,color:#155724; classDef srv fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef warn fill:#f8d7da,stroke:#dc3545,color:#721c24; class A,B,F app; class C,D,E srv; class G warn;

CODE You resend the whole transcript each turn

The client owns messages. Each turn appends and resends everything — the server remembers nothing between calls.

# The API is stateless — send the full conversation history each time.
import anthropic
client = anthropic.Anthropic()

messages = []  # ← YOU own this. The server keeps no copy.

def send(user_text):
    messages.append({"role": "user", "content": user_text})
    resp = client.messages.create(
        model="claude-opus-4-8",
        max_tokens=1024,
        messages=messages,                       # resend the ENTIRE list
    )
    reply = next(b.text for b in resp.content if b.type == "text")
    messages.append({"role": "assistant", "content": reply})
    print(resp.usage.input_tokens)          # grows every turn — history accumulates
    return reply

send("My name is Alice.")
send("What's my name?")  # works ONLY because turn 1 is still in messages[]

TRADE-OFFS Client-owned state

✅ You own the history

  • Full control: edit, branch, replay, persist, or prune however you like
  • No hidden server state to reason about or expire
  • Enables the 3.2–3.5 strategies — they operate on your list

⚠️ You own the history

  • Tokens (and cost + latency) grow every single turn
  • Nobody trims for you — unmanaged chats hit the window limit
  • You must persist state yourself to survive a restart
Because state is client-side, prompt caching pays off: keep the stable prefix (frozen system prompt, deterministic tools) first so each resend reuses the cache instead of reprocessing it at full price.

EXAM TRAPS

Trap: "The API remembers the conversation for me if I pass a session/thread ID." → The Messages API is stateless. There is no session ID; you resend the full messages array every turn.
Trap: "Claude 'forgot' what I told it earlier." → It never had it unless that turn is still in the messages you sent. Statelessness means dropped turns are genuinely gone.
Trap: Assuming only the new message is billed. → Everything in the request — full history, system prompt, tool defs, images — counts as input tokens every turn. That's why long chats get expensive and eventually overflow.
Trap: Confusing claude.ai with the API. → The chat product may roll context on a FIFO basis for you; the raw API does not. On the API, context management is your job.

PRACTICE

Q1 · What does "the Messages API is stateless" mean in practice?

The server keeps no memory of prior calls. To hold a conversation you must resend the entire history (system prompt + all prior messages + tool results) on every request. Response = f(system, tools, messages), with no server-side session.

Q2 · Why does a long conversation get more expensive and eventually fail?

Turns accumulate — previous turns are preserved completely, so input tokens grow every turn. Since all of it is re-sent and re-billed, cost and latency climb, and once input approaches the context-window limit you get a 400 "prompt is too long" (or model_context_window_exceeded mid-generation).

Q3 · Who is responsible for trimming or compressing the transcript, and why?

You are — the client owns the history. Because the API holds no state, the model can't prune for you. The 3.2–3.5 strategies (truncate, summarize, context-edit, external memory) are all things your application does to the messages list it controls.