CONCEPT Replace old turns with an LLM-generated summary
Truncation (3.2) throws early turns away. Summarization — also called
compaction — keeps their meaning: you feed the old turns to the model, ask it to write
a compact summary, then replace those turns with the summary in your messages list.
The transcript shrinks, but the gist survives: decisions made, facts established, the user's
goal, where the work stands.
A typical pattern: keep the last few turns verbatim (recency matters), summarize everything older into a single block, and prepend that block so the model still "remembers" the arc of the conversation. You can do this client-side (your own summarize-then-splice loop) or lean on Anthropic's server-side compaction, a beta feature that automatically summarizes earlier context on the server as the conversation approaches the context-window limit.
compact-2026-01-12, on Fable 5 / Opus 4.8 /
4.7 / 4.6 / Sonnet 5 / 4.6) returns a compaction block in the response. Critical: append the
full response.content — not just the text — back to messages. The API uses that block
to replace the compacted history on the next request; keep only the text and you silently lose the compaction
state.
CONCEPT The trade-off vs. raw truncation
Summarization sits between "keep everything" (accurate but overflows) and "drop the oldest" (cheap but forgets). Versus raw truncation:
| Truncation (3.2) | Summarization / Compaction (3.3) | |
|---|---|---|
| Old context | Deleted — gone | Compressed — gist retained |
| Extra cost | None | An extra LLM call to summarize |
| Fidelity | Perfect for kept turns, zero for dropped | Lossy: details/exact wording can blur |
| Best when | Only recency matters | Early goals/decisions must survive |
DIAGRAM Old turns collapse into one summary block
(old history)"] -->|"feed to model:
'summarize this'"| SUM["📝 Summary block
decisions · facts · goal · state"] SUM --> NEW["messages = [ summary ] + recent turns"] RECENT["Turns k+1…n
(kept verbatim)"] --> NEW NEW --> SEND["Send: smaller transcript,
gist preserved"] OLD -.->|"contrast: truncation would
DELETE turns 1…k"| DROP["🗑️ gist lost"] classDef old fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef good fill:#d4edda,stroke:#28a745,color:#155724; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class OLD old; class SUM,NEW,SEND,RECENT good; class DROP bad;
CODE Client-side summarize-and-splice
When the transcript gets big, summarize the old turns with a cheap call, then replace them with the summary and keep the recent tail verbatim.
# Compress old turns into a summary; keep recent turns verbatim. KEEP_RECENT = 6 def compact(messages): if len(messages) <= KEEP_RECENT: return messages old, recent = messages[:-KEEP_RECENT], messages[-KEEP_RECENT:] summary = client.messages.create( model="claude-opus-4-8", max_tokens=1024, system="Summarize this conversation. Preserve decisions, facts, " "the user's goal, and open tasks. Be faithful; do not invent.", messages=old, ).content[0].text # Replace the old turns with ONE summary block, keep the recent tail. return [ {"role": "user", "content": f"[Summary of earlier conversation]\n{summary}"}, *recent, ] # Server-side alternative (beta): let the API compact for you. # resp = client.beta.messages.create( # betas=["compact-2026-01-12"], model="claude-opus-4-8", max_tokens=1024, # context_management={"edits": [{"type": "compact_20260112"}]}, messages=messages) # messages.append({"role": "assistant", "content": resp.content}) # keep the compaction block!
TRADE-OFFS Compress vs. drop vs. keep
✅ Summarization / compaction
- Keeps the gist of the whole conversation, not just the tail
- Lets a session run far past the raw window limit
- Server-side compaction automates it — no client bookkeeping
⚠️ What it costs
- An extra model call (latency + tokens) to produce the summary
- Lossy: exact details can blur or be hallucinated
- Rewrites the prefix → invalidates prompt cache from that point
EXAM TRAPS
response.content including the compaction block; the API needs it to replace
the compacted history next turn. Keep only the text and you lose the compaction state.PRACTICE
Q1 · How does summarization differ from truncation, and what's the core trade-off?
Truncation drops old turns; summarization replaces them with an LLM-generated summary that preserves the gist (decisions, facts, goal, state). The trade-off: you spend an extra model call and accept some fidelity loss in exchange for not forgetting early context.
Q2 · With Anthropic's server-side compaction, what must you preserve on each turn?
The full response.content — including the compaction block — must be
appended back to messages. The API uses that block to replace the compacted history on the next
request. Appending only the text string silently loses the compaction state.
Q3 · Why isn't a summary a safe place for exact facts, and what fixes that?
Summaries are lossy: exact numbers, names, and verbatim requirements can be smoothed over or hallucinated. Write those to durable external memory (3.5) — files Claude reads back exactly — while using compaction to keep the active transcript small.