CONCEPT Every tool definition is input tokens on every request
Tools aren't free. The entire tools array — every name, description, and schema
— is serialized into the request as input tokens, and it's re-sent on every turn of a
conversation. On top of that, the API injects a hidden tool-use system prompt whenever
tools is present. So a large tool set costs context twice over: bigger prompts (money and latency)
and a harder selection problem for the model.
| What consumes tokens | Where |
|---|---|
Tool names + descriptions + input_schema | The tools parameter, every request |
| Auto-injected tool-use system prompt | Added whenever tools is set (hundreds of tokens; varies by model) |
tool_use blocks | Assistant output + resent in history |
tool_result blocks | Your follow-up request + resent in history |
input_examples (if used) | ~20–50 tokens simple, ~100–200 complex |
CONCEPT Keep the active set small; cache the tools block
Two levers manage the cost:
- Prune to the active set. Expose only the tools relevant to the task. If you have a huge
library, use the tool search tool with
defer_loadingso schemas load on demand instead of all sitting in context — this is where namespacing (2.4) pays off. - Prompt-cache the tools block. The rendered order is
tools→system→messages, so acache_controlbreakpoint on the last tool (or last system block) caches the whole tool set. On cache reads you pay roughly a tenth of the input price for that prefix — the big fixed token cost is paid once, not per turn.
tool_choice alone keeps the tools/system
cache — only the messages cache is invalidated.DIAGRAM Cost of a growing tool set — and the levers
defer_loading"] FIX -->|"stable tools block"| CACHE["🟢 Prompt-cache tools:
fixed cost paid once"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef info fill:#cfe2ff,stroke:#0d6efd,color:#084298; class PRUNE ok; class CACHE ok; class SEARCH info;
CODE Cache the tools block with cache_control
Put a cache_control breakpoint on the last tool definition. Because render order is
tools → system → messages, this caches the full tool set; later requests
with the same tools read it back at ~0.1× input price instead of re-billing every definition.
import anthropic client = anthropic.Anthropic() tools = [ # ... earlier tools ... { "name": "get_weather", "description": "Get the current weather in a given location. ...", "input_schema": { "type": "object", "properties": { "location": { "type": "string" } }, "required": ["location"] }, # Breakpoint on the LAST tool → caches the whole tools block. "cache_control": { "type": "ephemeral" }, }, ] resp = client.messages.create( model="claude-opus-4-8", max_tokens=1024, tools=tools, messages=[{"role": "user", "content": "Weather in Paris?"}], ) # First call writes the cache (~1.25x). Repeat calls with the SAME tools # read it back cheaply: print(resp.usage.cache_read_input_tokens) # > 0 on a cache hit
TRADE-OFFS Small active set vs. everything loaded
✅ Small, cached active tool set
- Lower per-turn token cost and latency
- Easier selection — fewer lookalikes
- Tools block cached: fixed cost paid once
⚠️ Every tool loaded, uncached
- Whole library billed on every turn
- More selection ambiguity as it grows
- Any tool-set edit invalidates the cache entirely
defer_loading:
schemas are appended on demand rather than swapped in, which also preserves the prompt cache.EXAM TRAPS
tool_choice can change without losing the tools/system cache.defer_loading so schemas load on demand instead of all sitting in context.PRACTICE
Q1 · What exactly does an unused tool cost you on each request?
Its full name + description + input_schema serialized as input tokens, re-sent every turn, plus a share of the auto-injected tool-use system prompt — and it adds to the selection difficulty. Unused tools are pure overhead in both token cost and accuracy.
Q2 · How do you make a large, stable tool set cheap to re-send?
Prompt-cache the tools block — put cache_control on the last tool
definition. Since render order is tools → system → messages, this caches the whole set; repeat requests read it
at ~0.1× input price. The block must stay byte-stable or the cache is invalidated.
Q3 · You have thousands of tools. What's the token-efficient approach?
Use the tool search tool with defer_loading: only relevant schemas load on
demand rather than all sitting in context. Namespaced names (2.4) make that on-demand matching reliable, and
appended schemas preserve the prompt cache.