CONCEPT Every token is billed and read — on both sides
You pay for input and output tokens, and both slow the call (7.3). Token efficiency is the one lever that helps every request regardless of model, cache, or batch: cut the tokens and you cut cost and latency at once. The moves fall into two buckets — shrink what you send (prompts, context, tool results) and shrink what you ask the model to produce (concise, structured, capped output).
| Pattern | Side | What it cuts |
|---|---|---|
| Concise prompts — drop boilerplate & repetition | Input | Wasted instruction tokens |
| Don't dump huge tool results | Input | Return only the fields needed; filter/summarize first |
| Ask for concise output | Output | Prose the caller never reads |
| Structure over prose (JSON / labels) | Output | Framing words around the actual answer |
Cap max_tokens | Output | Hard ceiling — model can't run long |
CONCEPT The biggest waste is uncontrolled context growth
In agent loops the input grows every turn — old tool results, verbose observations, and re-sent history pile
up, and you pay for all of it on every subsequent call. Two habits keep it lean: don't dump
raw tool output into context (return the 3 fields the model needs, not a 5,000-token API blob), and
ask for structured, minimal responses so the assistant turns you re-send stay small. Pair
this with a capped max_tokens so a single call can't balloon, and with prompt caching (7.5) so the
stable part you must re-send is billed at ~0.1×.
search() tool that returns titles +
IDs beats one that returns full documents — the model asks for the full doc only when it actually needs it.DIAGRAM Trim both sides of the call
(no boilerplate)"] -.-> IN P2["filtered tool results
(not raw dumps)"] -.-> IN P3["cached stable prefix"] -.-> IN P4["ask for concise output"] -.-> OUT P5["structure over prose"] -.-> OUT P6["cap max_tokens"] -.-> OUT CALL --> WIN["🟢 Lower cost + lower latency"] classDef lever fill:#d4edda,stroke:#28a745,color:#155724; classDef win fill:#d4edda,stroke:#28a745,color:#155724; class P1 lever; class P2 lever; class P3 lever; class P4 lever; class P5 lever; class P6 lever; class WIN win;
CODE Structure + cap beats free-form prose
Ask for a small structured answer and cap the length — the model returns the data, not a paragraph.
# WASTEFUL: open-ended prose, no ceiling → pays for framing words. # messages=[{"role":"user","content":"Tell me everything about this ticket"}] # EFFICIENT: structured output + a hard cap. resp = client.messages.create( model="claude-haiku-4-5", max_tokens=128, # ceiling — can't run long output_config={"format": { "type": "json_schema", "schema": { "type": "object", "properties": { "category": {"type": "string"}, "priority": {"type": "string", "enum": ["low", "high"]}, }, "required": ["category", "priority"], "additionalProperties": False, }, }}, messages=[{"role": "user", "content": f"Classify: {ticket}"}], ) # returns {"category": "...", "priority": "..."} — no prose to pay for
TRADE-OFFS Terse vs. useful
✅ Aggressive token trimming
- Lower cost on every call, input and output
- Faster (fewer tokens to read and generate)
- Structured output is easy to parse
- Keeps agent context from ballooning
⚠️ Cutting too far
- Stripping needed context degrades answers
- Over-tight
max_tokenstruncates output - Forcing structure on a genuinely open task can lose nuance
EXAM TRAPS
max_tokens huge "so it never truncates." → An uncapped
output can run far longer than needed. Set a ceiling sized to the task; raise it only when genuinely
truncated.PRACTICE
Q1 · Name two input-side and two output-side token-efficiency patterns.
Input: concise prompts (drop boilerplate) and not dumping raw tool results (filter to
needed fields). Output: ask for concise/structured answers (JSON or a label instead of prose) and cap
max_tokens.
Q2 · Why does structured output usually save more than a "be brief" instruction?
Structured output (JSON, an enum label) carries the answer with almost no surrounding framing words, and output tokens are billed at the higher rate. It also parses reliably downstream, avoiding a re-ask.
Q3 · In an agent loop, why is dumping a huge tool result especially costly?
Because history is re-sent every turn — a 5,000-token blob is paid for on this call and every subsequent one. Returning only the needed fields keeps the re-sent context small across the whole run.