D1 · Agentic Architecture › Cluster 1 · Foundations

1.4 · Statelessness & Conversation State

The API remembers nothing — "memory" is you replaying history

CONCEPT The API has no memory

POST /v1/messages is stateless. The server keeps no session, no conversation, no memory between calls. Each request is judged entirely on what you send in it: the system prompt and the full messages[] array. When the response comes back, the server forgets everything.

🔑 There is no "conversation ID" and no server-side session to attach to. What feels like "the model remembering the last turn" is you resending the last turn. Memory = replay.

So a multi-turn conversation is a discipline you implement: keep a growing messages[] list on your side, and on every call resend all of it plus the same system prompt. Turn N is a fresh request that happens to contain turns 1 … N−1 verbatim.

AssumptionReality
"The model remembers what I said earlier."Only because you resent it in messages[].
"I open a session and keep chatting."No sessions. Every call is independent and self-contained.
"The server stores history for me."You store it. The server reads it and forgets it.
"Longer chats are free once started."Every turn resends the whole history → input tokens grow each turn.
⚠️ Cost consequence: because you resend everything, input tokens grow with the conversation length. A 30-turn chat pays to re-read turns 1–29 on turn 30. Managing this is the job of pruning & compaction — see 7.5.

DIAGRAM Turn N resends turns 1..N-1

sequenceDiagram participant App as 🧑 Your App (holds messages[]) participant API as 🟣 /v1/messages (stateless) Note over App: messages = [u1] App->>API: system + [u1] API-->>App: a1 Note over App: messages = [u1, a1, u2] App->>API: system + [u1, a1, u2] (resend all) API-->>App: a2 Note over App: messages = [u1, a1, u2, a2, u3] App->>API: system + [u1, a1, u2, a2, u3] (resend all again) API-->>App: a3 Note over API: Server keeps NOTHING between calls

CODE A ConversationManager that replays history

# "Memory" is just: keep messages[] and resend everything each turn.
import anthropic

class ConversationManager:
    def __init__(self, system):
        self.client = anthropic.Anthropic()
        self.system = system
        self.messages = []               # YOU own the history

    def send(self, user_text):
        # 1) append the new user turn
        self.messages.append({"role": "user", "content": user_text})

        # 2) resend system + the ENTIRE history — stateless server
        resp = self.client.messages.create(
            model="claude-opus-4-8",
            max_tokens=1024,
            system=self.system,
            messages=self.messages,
        )

        # 3) append the whole assistant turn so the next call replays it (see 1.6)
        self.messages.append({"role": "assistant", "content": resp.content})
        return resp

chat = ConversationManager(system="You are a helpful tutor.")
chat.send("My name is Basant.")
chat.send("What's my name?")   # works ONLY because turn 1 was resent
The model "knows" the name on turn 2 purely because send() resent the earlier turns. Drop the history and the model is a blank slate again.

TRADE-OFFS Stateless replay vs. a server session

✅ Statelessness buys you

  • Total control — you decide exactly what the model sees each turn
  • Trivial to scale & retry; any request is self-contained
  • Easy to edit, prune, or branch history on your side

⚠️ You must handle

  • Storing and resending the full history yourself
  • Rising input-token cost as the chat grows (→ 7.5)
  • Context-window limits over long conversations (needs compaction)

EXAM TRAPS

Trap: Assuming a server-side session or "conversation ID." → There is none. The API is stateless; every call must carry the full system + messages[]. If you don't resend it, the model doesn't have it.
Trap: "The model forgot the earlier turn" as an API bug. → It's not a bug — the earlier turn wasn't in the messages[] you sent. Memory is your responsibility.
Trap: Assuming long conversations don't add cost once started. → Each turn resends the whole history, so input tokens grow every turn. Long chats need pruning/compaction to stay affordable and within the context window (7.5).

PRACTICE

Q1 · A user says their name in turn 1; in turn 5 the model doesn't know it. Where's the bug?

In your state handling, not the API. The turn-1 message wasn't included in the messages[] you sent on turn 5 (you dropped or truncated it). The stateless server only knows what you resend, so you must carry the relevant history forward.

Q2 · Why does a 50-turn conversation cost more per turn than a 2-turn one?

Because statelessness forces you to resend the full history every call, so input tokens grow with conversation length. Turn 50 re-reads turns 1–49. This is exactly why pruning and compaction (7.5) exist — to bound the resent context.

Q3 · Is there a "conversation ID" you can pass to resume a chat server-side?

No. The Messages API keeps no session. To "resume," you reload your own stored messages[] and resend it. All conversation state lives on your side.