D4 · Prompt Engineering › Cluster 5 · Chain of Thought & Extended Thinking

5.1 · Let Claude Think (CoT) — When It Helps

Reasoning before the answer

CONCEPT Ask for reasoning, get accuracy

Chain-of-thought (CoT) prompting means asking Claude to reason step by step before it commits to an answer. Instead of demanding the final result in one leap, you invite the model to externalize intermediate steps — the arithmetic, the case analysis, the logical chain. For complex, multi-step tasks — math, logic puzzles, financial or legal analysis, multi-hop questions — this reliably improves accuracy, because the model works the problem instead of pattern-matching to a plausible-looking guess.

🔑 The core intuition: giving the model room to reason in tokens before the answer lets it catch its own mistakes mid-derivation. The answer is only as good as the reasoning that produced it.

CoT comes in three levels of increasing structure:

LevelHow you promptUse when
Basic"Think step by step."Quick uplift; you don't need to parse the reasoning out.
GuidedName the steps: "First compute X, then compare to Y, then decide."You know the reasoning shape and want to steer it.
StructuredReason in <thinking>, answer in <answer> tags.You need to separate and strip reasoning from the shipped output (see 5.2).

CONCEPT When it helps — and when it's wasted

CoT is not free: every reasoning step is output tokens and latency. The gain is real on hard problems and roughly zero on easy ones. Classification, lookups, format conversion, sentiment tagging — a direct answer is faster, cheaper, and just as correct. Reserve CoT for tasks where a naive one-shot answer is genuinely error-prone.

Use CoT when: multi-step math, logical deduction, analysis that weighs several factors, or anything where you'd want a human to "show their work."
Skip CoT when: the task is a single lookup, a classification into known labels, or a mechanical transform. Reasoning here only adds tokens and latency with no accuracy payoff.

DIAGRAM Route the task by complexity

flowchart TD A["Incoming task"] --> Q{"Multi-step or
error-prone?"} Q -->|"no: lookup / classify / convert"| DIRECT["🟢 Answer directly
fewer tokens, lower latency"] Q -->|"yes: math / logic / analysis"| COT["🔵 Prompt for step-by-step
reasoning (CoT)"] COT --> STRUCT{"Need to strip
the reasoning?"} STRUCT -->|"yes"| TAGS["🟠 Structured CoT:
<thinking> then <answer> (5.2)"] STRUCT -->|"no"| PLAIN["Plain 'think step by step'"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef mid fill:#cfe2ff,stroke:#0d6efd,color:#084298; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class DIRECT ok; class COT mid; class TAGS gate;

CODE BAD → GOOD: give the model room to reason

A bare "what's the answer" on a multi-step word problem invites a fast wrong guess. Asking for the steps first fixes it — same model, same question, better answer.

# BAD — one-shot, no room to reason. Error-prone on multi-step math.
system = "You are a financial analyst."
user   = "A bond pays $50/yr for 3 yrs then $1050. At 6% discount, what's its price?"
# → model blurts a number; may fumble the discounting.

# GOOD — ask for step-by-step reasoning first.
user = """A bond pays $50/yr for 3 yrs then $1050. At a 6% discount rate,
what is its present value?

Think step by step: discount each cash flow, then sum them.
Show your work, then give the final price."""
# → model discounts each flow, sums, and states the price — far more reliable.
⚠️ CoT surfaces the reasoning in the visible response. If you don't want that text shipped to the end user, use the structured <thinking>/<answer> pattern in 5.2 so you can strip it — or use native extended thinking (5.3), which keeps reasoning in separate blocks.

TRADE-OFFS Reasoning vs. a direct answer

✅ Prompt for CoT

  • Higher accuracy on math, logic, multi-factor analysis
  • Model catches its own mistakes mid-derivation
  • Reasoning is inspectable and debuggable

⚠️ Costs of CoT

  • More output tokens → higher cost
  • Higher latency (reasoning streams before the answer)
  • No benefit on simple tasks — pure overhead

EXAM TRAPS

Trap: "Always add 'think step by step.'" → CoT helps complex tasks; on lookups and classification it just burns tokens and latency for no accuracy gain.
Trap: Assuming CoT is free. → It costs output tokens and latency. The reasoning is generated text, billed and streamed like any other output.
Trap: Confusing CoT with a longer answer. → CoT is about reasoning before the answer, not verbosity. The goal is a correct final result, with the steps as scaffolding.
Trap: Thinking prompted CoT and native extended thinking are the same thing. → Prompted CoT is you engineering the prompt; extended thinking (5.3) is a native model capability with its own thinking parameter and separate reasoning blocks.

PRACTICE

Q1 · You're classifying support tickets into 5 known categories. Should you add CoT?

No. Classification into a fixed label set is a simple task — a direct answer is faster and just as accurate. CoT here only adds tokens and latency. Reserve it for multi-step reasoning like math or analysis.

Q2 · Why does "think step by step" improve accuracy on a multi-step math problem?

It gives the model room to externalize intermediate steps in tokens before committing to an answer, so it works the problem instead of guessing a plausible number — and it can catch its own errors mid-derivation.

Q3 · Name the two costs of prompted CoT and one task where they're not worth paying.

Extra output tokens and higher latency. They're not worth it on simple tasks — e.g. a single lookup, a format conversion, or classification into known labels — where a direct answer is equally correct.