CCA-P start here · Trap gym · Trap 10

Cache it, don't cut it

D2 · Models, Prompting & ContextGuide lines 2.5, 4.5

At a glance: When the same long text goes with every request, put it first and cache it; cutting it down or switching to a smaller model gives up quality for a saving that caching gives for free.

Many assistants send the same long block on every request: a system prompt, a policy, a rulebook, tool definitions. Only the user's short message changes. Paying full price for that same block thousands of times is the waste, and the tempting fixes are to cut it (send less) or downsize (use a smaller model). Both risk wrong answers, because the content was there for a reason.

Prompt caching attacks the waste directly. If the start of the prompt (the prefix) is identical on every request, Claude reads it from a cache: much cheaper, and the first words of the reply arrive sooner. The rule is ordering: static content first, anything that changes per request after it. One changing detail near the top, such as a date, a name or a different tool order, breaks the cache from that point on.

Caching is not always the answer. If nothing repeats, there is nothing to reuse. If the reference is far too big to send at all, retrieve only the relevant part. And if part of the block is genuinely never needed, removing it is fine: the trap is cutting content the answers need.

Change one fact

A pension provider's member assistant sends the same 10,000-token system prompt and scheme rulebook on every request, followed by the member's short question. Members wait on screen for each reply. Cost and reply time are both too high, and any part of the rulebook may be needed for a correct answer. What should the team change?

Words in the stem that signal this trap

Rule to remember: A long text repeated on every request is a caching problem, not a cutting problem: static content first, then cache it. Cut only what the answers never need, and retrieve when the reference is too big to send.

Practice

Practice 10.1 · Choose one.

A law firm's contract assistant sends the same 20,000-token clause library first in every request, then the lawyer's question. Prompt caching is on, and the prefix never changes. Lawyers send a question about every fifteen minutes, longer than the default cache lifetime, so most requests miss the cache. What should the architect change?

  1. Choose the longer cache lifetime, so the prefix stays cached between requests.
  2. Shorten the clause library by removing the clauses that lawyers cite least often.
  3. Move the lawyer's question so it comes after the static clause library.
  4. Switch to a smaller model so that each uncached request costs less.
Show answer

Answer: A. The deciding fact: the prefix is identical and correctly placed, but requests arrive further apart than the cache lasts. A longer cache lifetime keeps it warm between requests.

  • A ✅ Choose the longer cache lifetime, so the prefix stays cached between requests.
  • B ❌ Works to cut tokens, but it removes clauses some answers need, and the misses would continue.
  • C ❌ Works when variable content sits in the prefix; here the order is already right.
  • D ❌ Works if the eval holds, but it leaves the real cause, an expiring cache, untouched.
Practice 10.2 · Choose one.

An appliance maker's support assistant sends a cached 14,000-token prefix with every chat. Half of it is manuals for products the company stopped supporting two years ago, and policy says the assistant must not advise on them. Costs are acceptable, but the assistant sometimes quotes the old manuals. What should the architect do?

  1. Keep the prefix as it is, since caching already makes the old manuals cheap to send.
  2. Add a system-prompt rule telling the assistant to ignore the unsupported manuals.
  3. Move the unsupported manuals to a retrieval index the assistant searches when needed.
  4. Remove the unsupported manuals from the prefix and cache the part that remains.
Show answer

Answer: D. The deciding fact: the old manuals are never needed and must not be used. Here cutting is right, because the removed text is content no answer should draw on.

  • A ❌ Works for content that is needed; here cheap content still misleads the assistant.
  • B ❌ Works sometimes, but a prompt rule is probabilistic and the manuals stay in front of the model.
  • C ❌ Works for large content needed now and then; here the content should never be used at all.
  • D ✅ Remove the unsupported manuals from the prefix and cache the part that remains.
Practice 10.3 · Choose one.

A car-rental firm's booking assistant sends the same system prompt, 30 tool definitions and a fare-rules document on every request, then the customer's message. Prompt caching is on, but the hit rate is low. Logs show the tool definitions are added in a different order on each request. What should the architect change?

  1. Shorten every tool description so that each uncached request costs less.
  2. Send the tool definitions in one fixed order on every request.
  3. Turn caching off, since a prefix that keeps changing cannot be cached.
  4. Load the tools through tool search so fewer definitions are sent.
Show answer

Answer: B. The deciding fact: the content is the same each time, only the order varies, and any change early in the prefix breaks the cache. A fixed order makes the prefix identical.

  • A ❌ Works to trim tokens, but descriptions help tool choice, and the cache would still miss.
  • B ✅ Send the tool definitions in one fixed order on every request.
  • C ❌ Works when the content truly differs per request; here only the order differs.
  • D ❌ Works for hundreds of tools with a few used per task; 30 stable tools are cheaper cached.

Next trap: Agree it in numbers → · All traps