CONCEPT Ask for reasoning, get accuracy
Chain-of-thought (CoT) prompting means asking Claude to reason step by step before it commits to an answer. Instead of demanding the final result in one leap, you invite the model to externalize intermediate steps — the arithmetic, the case analysis, the logical chain. For complex, multi-step tasks — math, logic puzzles, financial or legal analysis, multi-hop questions — this reliably improves accuracy, because the model works the problem instead of pattern-matching to a plausible-looking guess.
CoT comes in three levels of increasing structure:
| Level | How you prompt | Use when |
|---|---|---|
| Basic | "Think step by step." | Quick uplift; you don't need to parse the reasoning out. |
| Guided | Name the steps: "First compute X, then compare to Y, then decide." | You know the reasoning shape and want to steer it. |
| Structured | Reason in <thinking>, answer in <answer> tags. | You need to separate and strip reasoning from the shipped output (see 5.2). |
CONCEPT When it helps — and when it's wasted
CoT is not free: every reasoning step is output tokens and latency. The gain is real on hard problems and roughly zero on easy ones. Classification, lookups, format conversion, sentiment tagging — a direct answer is faster, cheaper, and just as correct. Reserve CoT for tasks where a naive one-shot answer is genuinely error-prone.
DIAGRAM Route the task by complexity
error-prone?"} Q -->|"no: lookup / classify / convert"| DIRECT["🟢 Answer directly
fewer tokens, lower latency"] Q -->|"yes: math / logic / analysis"| COT["🔵 Prompt for step-by-step
reasoning (CoT)"] COT --> STRUCT{"Need to strip
the reasoning?"} STRUCT -->|"yes"| TAGS["🟠 Structured CoT:
<thinking> then <answer> (5.2)"] STRUCT -->|"no"| PLAIN["Plain 'think step by step'"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef mid fill:#cfe2ff,stroke:#0d6efd,color:#084298; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class DIRECT ok; class COT mid; class TAGS gate;
CODE BAD → GOOD: give the model room to reason
A bare "what's the answer" on a multi-step word problem invites a fast wrong guess. Asking for the steps first fixes it — same model, same question, better answer.
# BAD — one-shot, no room to reason. Error-prone on multi-step math. system = "You are a financial analyst." user = "A bond pays $50/yr for 3 yrs then $1050. At 6% discount, what's its price?" # → model blurts a number; may fumble the discounting. # GOOD — ask for step-by-step reasoning first. user = """A bond pays $50/yr for 3 yrs then $1050. At a 6% discount rate, what is its present value? Think step by step: discount each cash flow, then sum them. Show your work, then give the final price.""" # → model discounts each flow, sums, and states the price — far more reliable.
<thinking>/<answer> pattern in
5.2 so you can strip it — or use native extended thinking (5.3), which keeps reasoning in separate blocks.TRADE-OFFS Reasoning vs. a direct answer
✅ Prompt for CoT
- Higher accuracy on math, logic, multi-factor analysis
- Model catches its own mistakes mid-derivation
- Reasoning is inspectable and debuggable
⚠️ Costs of CoT
- More output tokens → higher cost
- Higher latency (reasoning streams before the answer)
- No benefit on simple tasks — pure overhead
EXAM TRAPS
thinking parameter and separate reasoning blocks.PRACTICE
Q1 · You're classifying support tickets into 5 known categories. Should you add CoT?
No. Classification into a fixed label set is a simple task — a direct answer is faster and just as accurate. CoT here only adds tokens and latency. Reserve it for multi-step reasoning like math or analysis.
Q2 · Why does "think step by step" improve accuracy on a multi-step math problem?
It gives the model room to externalize intermediate steps in tokens before committing to an answer, so it works the problem instead of guessing a plausible number — and it can catch its own errors mid-derivation.
Q3 · Name the two costs of prompted CoT and one task where they're not worth paying.
Extra output tokens and higher latency. They're not worth it on simple tasks — e.g. a single lookup, a format conversion, or classification into known labels — where a direct answer is equally correct.