A bigger model is not the fix
D2 · Models, Prompting & ContextGuide lines 2.1, 2.4, 3.8
A context window is how much text Claude can read in one request. A bigger window raises that limit, but it does not make the extra text free: every token still costs money and time, and a model given hundreds of pages to find one rule pays less attention to that rule. A more capable model reasons better, but it is slower and dearer per token, and it cannot use a fact it was never given.
So when quality, cost or speed is poor, look at what goes into the request first. If only a small part of a large source is needed each time, retrieve that part. If the content is small, stable and needed on almost every request, keep it in the prompt, put it first, and cache it, so repeated requests reuse it cheaply. If the needed context is all there and the prompt is clear, and the task still fails on hard reasoning, that is when a more capable model earns its price.
The same logic runs the other way: the smallest model that passes your evaluation is the right one for a simple task, and paying for the biggest one “to be safe” is waste.
Change one fact
A delivery firm's assistant answers drivers' questions about delivery rules. The whole 600-page driver handbook is pasted into every request. Answers are slow and expensive, and often miss the rule that applies. Most questions need one or two sections. A manager proposes moving to the most capable model. What should the architect do?
Words in the stem that signal this trap
- “the whole document is pasted into every request”
- “a manager proposes the most capable model” or “a larger context window”
- each task uses a small part of a large source
- small, stable content needed on every request (cache it instead)
- “traces show the right context and a clear prompt” (now a bigger model may win)
Practice
A law firm's assistant answers questions about one client's 120-page supply agreement, which fits easily in the context window. The team puts the question at the top of the prompt, then the whole agreement. Answers often miss definitions that sit deep in the document. A partner suggests a model with a larger context window. What should the architect try first?
- Move to a model with a larger context window and keep the prompt as it is.
- Split the agreement into chunks and retrieve the five closest to each question.
- Put the agreement first and the question last, and ask for supporting quotes.
- Switch to a smaller model so that each long request returns more quickly.
Show answer
Answer: C. The deciding fact: the agreement already fits, and the question sits above a long document. Document first, question last, and quoting the relevant passages first help the model use long inputs.
- A ❌ Works for documents that do not fit, but this one does, so more room changes nothing.
- B ❌ Works for large collections, but here it could split a term from the definition it relies on.
- C ✅ Put the agreement first and the question last, and ask for supporting quotes.
- D ❌ Works when a simple task is overpaying, but speed is not the problem here.
An engineering firm's agent checks structural calculations in design reports against building codes. Traces show the right code sections and full calculations are in context, and the prompt has been rewritten with clear steps and three worked examples. It still errs when one check chains several formulas together. What should the architect do?
- Add more building-code sections so the agent has wider background to draw on.
- Move to a model with a larger context window so whole reports fit in one request.
- Lower the temperature so that the agent's results are consistent from run to run.
- Run the failing cases on the most capable model, and move the task if they pass.
Show answer
Answer: D. The deciding fact: context is right and the prompt is clear, yet multi-step reasoning fails. Testing the same cases on a more capable model confirms a model mismatch.
- A ❌ Works when a needed section is missing, but traces show it is already there.
- B ❌ Works when content does not fit, but the reports already fit.
- C ❌ Works for consistency, but consistent output is not correct output.
- D ✅ Run the failing cases on the most capable model, and move the task if they pass.
A museum writes short alt-text descriptions for 40,000 archive photos a month. Staff check them in a weekly batch, so nothing is needed instantly. The pilot used the most capable model, and costs are over budget. Reviewers found nearly every description simple and accurate. What should the architect recommend?
- Test a smaller model on a reviewed sample, then run it in batches if it passes.
- Keep the most capable model and ask for a bigger budget, since access matters.
- Move to the smallest available model at once, since the descriptions are short.
- Send several photos per request to the most capable model to cut the call count.
Show answer
Answer: A. The deciding facts: the task is simple and nobody waits for results. A smaller model validated on reviewed cases, run through batch processing, keeps quality and cuts cost.
- A ✅ Test a smaller model on a reviewed sample, then run it in batches if it passes.
- B ❌ Works when the task is hard, but reviewers found it simple.
- C ❌ Works only if it passes an evaluation; switching without one risks quality.
- D ❌ Works a little, but it keeps paying top price for a simple task.