FRAME First ask: what kind of bloat is it?
"Context is too big" isn't one problem — it has several causes, and each has a different right fix. Diagnose the cause first, then apply the matching technique. Using the wrong one (e.g. summarising when you just needed to prune tool results) is how you break the conversation.
DECIDE Decision flow
flowchart TD
S["Context is bloating"] --> Q1{"What's the cause?"}
Q1 -->|"Re-sending a big STABLE prefix
is expensive"| C["Prompt caching"]:::a Q1 -->|"Old tool_result payloads dominate,
but the dialogue must stay"| E["Context editing
(clear_tool_uses)"]:::b Q1 -->|"The WHOLE conversation is
near the window limit"| K["Compaction
(compact)"]:::c Q1 -->|"Facts must survive
ACROSS sessions"| M["Memory tool"]:::d Q1 -->|"Simple 'keep last N turns'
is enough (few/no tools)"| T["Safe truncation
(sliding window)"]:::e Q1 -->|"Many/large tool schemas
or huge tool outputs"| L["Tool slimming
(tool search · lean results · PTC)"]:::f classDef a fill:#e0f2fe,stroke:#0369a1,color:#075985; classDef b fill:#dcfce7,stroke:#15803d,color:#14532d; classDef c fill:#fef3c7,stroke:#b45309,color:#7c2d12; classDef d fill:#ede9fe,stroke:#6d28d9,color:#4c1d95; classDef e fill:#ffe4e6,stroke:#be123c,color:#7f1d1d; classDef f fill:#f1f5f9,stroke:#475569,color:#334155;
is expensive"| C["Prompt caching"]:::a Q1 -->|"Old tool_result payloads dominate,
but the dialogue must stay"| E["Context editing
(clear_tool_uses)"]:::b Q1 -->|"The WHOLE conversation is
near the window limit"| K["Compaction
(compact)"]:::c Q1 -->|"Facts must survive
ACROSS sessions"| M["Memory tool"]:::d Q1 -->|"Simple 'keep last N turns'
is enough (few/no tools)"| T["Safe truncation
(sliding window)"]:::e Q1 -->|"Many/large tool schemas
or huge tool outputs"| L["Tool slimming
(tool search · lean results · PTC)"]:::f classDef a fill:#e0f2fe,stroke:#0369a1,color:#075985; classDef b fill:#dcfce7,stroke:#15803d,color:#14532d; classDef c fill:#fef3c7,stroke:#b45309,color:#7c2d12; classDef d fill:#ede9fe,stroke:#6d28d9,color:#4c1d95; classDef e fill:#ffe4e6,stroke:#be123c,color:#7f1d1d; classDef f fill:#f1f5f9,stroke:#475569,color:#334155;
🔑 The two most-confused options: context editing PRUNES stale tool results (keeps the
dialogue); compaction SUMMARISES the whole history (rewrites it). If the problem is specifically bloated
tool_results and the running dialogue must stay intact → context editing, every time.
TABLE The master cheat sheet
| Technique | Use when… | What it does | ⚠️ Integrity / reliability rule |
|---|---|---|---|
Token countingcount_tokens |
You need to decide whether/when to act | Measures exact, model-specific token size | Use the real endpoint — never estimate with tiktoken or "chars ÷ 4" (they under-count Claude tokens) |
Prompt cachingcache_control |
A large stable prefix is re-sent across many requests | Reuses the processed prefix → ~90% cheaper reads | Cuts cost, not footprint. Prefix must be byte-stable (a datetime/uuid silently invalidates it); put volatile content after the last breakpoint. Verify with cache_read_input_tokens |
Context editingclear_tool_uses_20250919 |
Old, no-longer-needed tool_result payloads dominate the window — but the dialogue must stay | Prunes stale tool results (optionally their inputs/thinking) server-side before the model sees them | Keeps message roles & the running dialogue intact — it is NOT summarisation. You keep the full history client-side; the API reports what it cleared |
Compactioncompact_20260112 |
The whole conversation is approaching the context-window limit | Summarises earlier history into a compaction block, server-side | MUST append response.content (not just .text) back to messages every turn — the compaction block carries the state; dropping it corrupts the next request |
Memory toolmemory_20250818 |
Facts must persist across sessions, not just within one conversation | Claude reads/writes a /memories dir you back with storage |
Lives outside the window (no bloat). Guard against path traversal; never store secrets/PII (it replays into every future session) |
| Safe truncation (client-side sliding window) |
A simple "keep the last N turns" suffices; few/no tools | Drops the oldest turns yourself | Cut only at a plain user turn so a tool_use never loses its tool_result; keep the first (instructions) turn. It invalidates the prompt cache (prefix changes) — prefer context editing when you only need to drop tool results |
| Tool slimming (design) |
Fat tool schemas or huge tool outputs are the bloat | Tool search + defer_loading; return lean results; programmatic tool calling |
Doesn't change roles/pairing — just keeps each turn small. Tool search appends schemas, so the cache is preserved |
CONTRAST The distinctions the exam probes
🟢 Context editing (prune)
- Targets stale tool_results specifically
- Dialogue & message structure untouched
- Lossless for the conversation (only junk removed)
- Use when: tool payloads bloat, dialogue must stay
🟠 Compaction (summarise)
- Condenses the whole history into a summary
- Lossy — the dialogue is rewritten
- Needs the
response.contentround-trip or state is lost - Use when: the entire conversation nears the window limit
Trap: "context is too big" ⇒ reach for compaction. Not always — if it's tool-result
bloat with a live dialogue, that's context editing. And prompt caching does not shrink the window at all
— it only makes re-sending cheaper.
STACK They combine
Real long-running agents use several at once: prompt-cache the stable prefix, context-edit stale tool results as the loop runs, compact if the whole thing still approaches the limit, and use memory for anything that must outlive the session. Keep tool schemas/results lean throughout.
🧑💻 Runnable demo of all of these (done correctly):
examples/context_bloat_techniques.py ·
theory: Cluster 3 — Managing Long Conversations and
3.4 Context Editing ·
3.3 Compaction ·
3.5 Memory Tool.