D2 · Tool Design & MCP › Cluster 2 · Designing Great Tools

2.5 · Token Efficiency & Tool-Set Size

Every tool costs context

CONCEPT Every tool definition is input tokens on every request

Tools aren't free. The entire tools array — every name, description, and schema — is serialized into the request as input tokens, and it's re-sent on every turn of a conversation. On top of that, the API injects a hidden tool-use system prompt whenever tools is present. So a large tool set costs context twice over: bigger prompts (money and latency) and a harder selection problem for the model.

What consumes tokensWhere
Tool names + descriptions + input_schemaThe tools parameter, every request
Auto-injected tool-use system promptAdded whenever tools is set (hundreds of tokens; varies by model)
tool_use blocksAssistant output + resent in history
tool_result blocksYour follow-up request + resent in history
input_examples (if used)~20–50 tokens simple, ~100–200 complex
🔑 Two costs scale with tool-set size: token cost (bigger prompt) and selection difficulty (more lookalike descriptions to disambiguate — see 2.3/2.4). Keeping the active tool set small helps both at once.

CONCEPT Keep the active set small; cache the tools block

Two levers manage the cost:

Caching only helps if the tools block is byte-stable. Adding, removing, or reordering a tool invalidates the entire cache (tools render at position 0). Serialize tools deterministically and don't churn the set mid-conversation. Note: changing tool_choice alone keeps the tools/system cache — only the messages cache is invalidated.

DIAGRAM Cost of a growing tool set — and the levers

flowchart TD N["N tools in context"] --> C1["↑ input tokens every turn"] N --> C2["↑ selection difficulty"] C1 --> FIX{"Manage the cost"} C2 --> FIX FIX -->|"expose only relevant tools"| PRUNE["🟢 Small active set"] FIX -->|"huge library"| SEARCH["🔵 Tool search +
defer_loading"] FIX -->|"stable tools block"| CACHE["🟢 Prompt-cache tools:
fixed cost paid once"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef info fill:#cfe2ff,stroke:#0d6efd,color:#084298; class PRUNE ok; class CACHE ok; class SEARCH info;

CODE Cache the tools block with cache_control

Put a cache_control breakpoint on the last tool definition. Because render order is tools → system → messages, this caches the full tool set; later requests with the same tools read it back at ~0.1× input price instead of re-billing every definition.

import anthropic
client = anthropic.Anthropic()

tools = [
    # ... earlier tools ...
    {
        "name": "get_weather",
        "description": "Get the current weather in a given location. ...",
        "input_schema": { "type": "object",
            "properties": { "location": { "type": "string" } },
            "required": ["location"] },
        # Breakpoint on the LAST tool → caches the whole tools block.
        "cache_control": { "type": "ephemeral" },
    },
]

resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    tools=tools,
    messages=[{"role": "user", "content": "Weather in Paris?"}],
)
# First call writes the cache (~1.25x). Repeat calls with the SAME tools
# read it back cheaply:
print(resp.usage.cache_read_input_tokens)   # > 0 on a cache hit

TRADE-OFFS Small active set vs. everything loaded

✅ Small, cached active tool set

  • Lower per-turn token cost and latency
  • Easier selection — fewer lookalikes
  • Tools block cached: fixed cost paid once

⚠️ Every tool loaded, uncached

  • Whole library billed on every turn
  • More selection ambiguity as it grows
  • Any tool-set edit invalidates the cache entirely
For thousands of tools, reach for tool search + defer_loading: schemas are appended on demand rather than swapped in, which also preserves the prompt cache.

EXAM TRAPS

Trap: "Tools are basically free — add all of them." → Every tool's name, description, and schema are input tokens on every request, plus an auto-injected tool-use system prompt. Keep the active set small.
Trap: Cost is only tokens. → A large tool set also raises selection difficulty. Trimming it improves accuracy and cost together.
Trap: "I cached the tools, so I can freely add/remove tools." → Any change to the tools block invalidates the whole cache (tools render at position 0). Keep it byte-stable; only tool_choice can change without losing the tools/system cache.
Trap: Loading thousands of tools directly. → Use the tool search tool with defer_loading so schemas load on demand instead of all sitting in context.

PRACTICE

Q1 · What exactly does an unused tool cost you on each request?

Its full name + description + input_schema serialized as input tokens, re-sent every turn, plus a share of the auto-injected tool-use system prompt — and it adds to the selection difficulty. Unused tools are pure overhead in both token cost and accuracy.

Q2 · How do you make a large, stable tool set cheap to re-send?

Prompt-cache the tools block — put cache_control on the last tool definition. Since render order is tools → system → messages, this caches the whole set; repeat requests read it at ~0.1× input price. The block must stay byte-stable or the cache is invalidated.

Q3 · You have thousands of tools. What's the token-efficient approach?

Use the tool search tool with defer_loading: only relevant schemas load on demand rather than all sitting in context. Namespaced names (2.4) make that on-demand matching reliable, and appended schemas preserve the prompt cache.