D1 · Agentic Architecture › Cluster 7 · Performance

7.5 · Conversation-State Management

Four ways to keep long runs inside the context window

CONCEPT Why state is your problem

The Messages API is stateless — you resend the full conversation every turn. So in a long agentic run, the transcript grows until it threatens the context limit, and every turn re-pays for all those tokens. Four techniques keep the window bounded; they compose, and the exam expects you to know when to use each.

TechniqueWhat it doesTrade-offUse when…
Pruning / windowing Drop old turns; keep a recent window. Simple & cheap, but you lose the dropped info outright. Only recent context matters (e.g. chat-style).
Compaction (beta) Server-side summarizes the conversation as it nears the context limit. Preserves gist; you must append response.content back to keep the compaction blocks. Long runs where older detail should survive as a summary.
Context editing Prunes stale tool_results and thinking blocks. Reclaims tokens from bulky intermediate data; keeps the dialogue. Tool-heavy runs with large, now-irrelevant tool outputs.
Memory tool Cross-session persistence via a /memories directory. Survives across conversations, not just within one. Facts must outlive a single run/session.
🔑 Pruning, compaction, and context editing manage state within one run; the Memory tool is the only one that persists across sessions (/memories).

DIAGRAM Choosing a state technique

flowchart TD A([Context growing]) --> Q1{"Need info to survive
across sessions?"} Q1 -->|Yes| M["🟣 Memory tool
/memories directory"] Q1 -->|No| Q2{"Bulky stale tool_results
/ thinking blocks?"} Q2 -->|Yes| C["🔵 Context editing
prune tool_results & thinking"] Q2 -->|No| Q3{"Must older detail
survive as a summary?"} Q3 -->|Yes| K["🟠 Compaction (beta)
append response.content back!"] Q3 -->|No| P["🟢 Prune / window
drop old turns"] classDef mem fill:#e2d4f0,stroke:#764ba2,color:#3d1a5b; classDef ce fill:#d1ecf1,stroke:#17a2b8,color:#0c5460; classDef cm fill:#ffe8cc,stroke:#d9480f,color:#7a2e08; classDef pr fill:#d4edda,stroke:#28a745,color:#155724; class M mem; class C ce; class K cm; class P pr;

CODE A windowed loop — and the compaction gotcha

Windowing is a few lines you own. Compaction is server-side, but it hands you compaction blocks in response.content that you must append back, or the next turn loses them.

# Stateless API → you resend everything. Keep a bounded window yourself.
def window(messages, keep=20):
    # Keep the first turn (task) + the most recent `keep` turns.
    return messages[:1] + messages[-keep:] if len(messages) > keep + 1 else messages

while True:
    resp = client.messages.create(
        model="claude-opus-4-8", max_tokens=4096,
        tools=TOOLS, messages=window(messages),
    )
    # ⚠️ With compaction (beta): append the WHOLE response.content back,
    # so server-generated compaction blocks are preserved for the next turn.
    messages.append({"role": "assistant", "content": resp.content})
    if resp.stop_reason == "end_turn": break
    # ...run chosen tool, append tool_result, loop
💡 Compaction is beta and server-side: you don't summarize yourself, but you must faithfully echo response.content back into messages so the compaction blocks it produced aren't dropped on the next call.

TRADE-OFFS Lossy speed vs. lossless recall

✅ Prune / window / context-edit

  • Cheapest, most predictable token savings
  • Context editing targets bulky tool_results/thinking specifically
  • You stay in control of exactly what's dropped

⚠️ Compaction / Memory

  • Compaction preserves gist but is lossy & needs the append-back discipline
  • Memory adds a persistence surface to manage (/memories)
  • More moving parts than a simple window

EXAM TRAPS

Trap: Using compaction but not appending response.content back. → You drop the compaction blocks and the next turn regresses. Always echo the full content back.
Trap: Reaching for the Memory tool to shrink one long conversation. → Memory is for cross-session persistence; within a run use pruning, context editing, or compaction.
Trap: Assuming the server "remembers" between calls. → The API is stateless; you resend state every turn, which is exactly why these techniques exist.
Trap: Pruning when you actually needed the old detail. → Windowing is lossy. If older detail must survive, use compaction (summary) or context editing (targeted), not a blunt drop.

PRACTICE

Q1 · A coding agent's context is dominated by huge past tool_result payloads it no longer needs. Which technique?

Context editing — it prunes stale tool_results and thinking blocks specifically, reclaiming those tokens while keeping the live dialogue intact.

Q2 · Users expect the assistant to recall preferences from last week's separate session. Which technique?

The Memory tool — cross-session persistence via a /memories directory. The other three only manage state within a single run.

Q3 · After enabling compaction, older facts vanish a turn later. What did the code forget to do?

Append the full response.content back into messages. Compaction is server-side and returns compaction blocks in the response; if you don't echo them back, they're lost on the next call.