CONCEPT Prompt engineering is the fastest, cheapest lever
Prompt engineering is the practice of shaping a model's behaviour by editing the words you send it — the instructions, context, examples, and structure of the prompt — rather than changing the model's weights. For most use cases it is the first lever you should reach for, because it is far faster and cheaper to iterate on than the alternatives.
Set against fine-tuning, the contrast is stark. A prompt edit gives you a new result in seconds; a fine-tune requires curating a training dataset, running a training job, and evaluating a new checkpoint — hours to days and real compute cost per iteration. Prompting also preserves the model's general capabilities (fine-tuning can narrow them), needs no labelled data, and adapts instantly when requirements change. That is why the standard guidance is: exhaust prompt engineering first, and only move to fine-tuning when prompting genuinely can't close the gap.
| Lever | Iteration speed | Cost | Reach for it when… |
|---|---|---|---|
| Prompt engineering | Seconds | ~None (just tokens) | Almost always — the default first move |
| Few-shot examples in prompt | Minutes | Extra input tokens | Format/tone/structure need pinning down |
| Model selection | Minutes | Per-token price delta | Latency or cost is the real problem, not quality |
| Fine-tuning | Hours–days | Data + training compute | Prompting truly can't reach the behaviour, and you have data |
CONCEPT When prompt engineering is not the answer
Prompting is powerful but not universal. Anthropic's own guidance is explicit: "Not every success criteria or failing eval is best solved by prompt engineering. For example, latency and cost can sometimes be more easily improved by selecting a different model." A prompt cannot make a model faster or cheaper — those are properties of the model tier you choose. If your failing metric is speed or price, the fix is a smaller/faster model (or fast mode), not a cleverer prompt.
Likewise, before you even start tuning, Anthropic assumes you have three things in place. Missing any of them means you're optimizing blind:
DIAGRAM Which lever fixes this failing eval?
edit the prompt — seconds"] Q -->|"Too slow / too expensive"| M["🔵 Change the model tier
not the prompt"] Q -->|"Durable capability prompting can't reach"| F{"Have data + eval budget?"} F -->|"yes"| FT["🟠 Fine-tune
hours–days, real cost"] F -->|"no"| P classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef alt fill:#cfe2ff,stroke:#0d6efd,color:#084298; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class P ok; class M alt; class FT gate;
CODE The same behaviour: prompt tweak vs. the fine-tune you avoided
A behaviour that looks like it needs training — "always answer as a terse SRE" — is usually one prompt away. This is the whole argument for reaching for prompting first: no dataset, no training run, instant result.
# ❌ The instinct: "the model won't behave, let's fine-tune." # → curate 1000s of examples, run a training job, eval a new checkpoint. # → hours-to-days, real compute cost, PER iteration. # ✅ The lever to try FIRST: a system prompt. New behaviour in seconds. resp = client.messages.create( model="claude-opus-4-8", max_tokens=1024, system=( "You are a senior SRE. Answer in at most 3 sentences. " "Lead with the action to take, then the one-line reason. " "No preamble, no pleasantries." ), messages=[{"role": "user", "content": "Disk is 95% full on prod-db-1. What now?"}], ) # Edit the system string → re-run → new behaviour. That loop is the point.
TRADE-OFFS Prompt engineering vs. fine-tuning
✅ Prompt engineering (start here)
- Iterate in seconds; no training pipeline
- No labelled dataset required
- Preserves the model's general capabilities
- Reversible — just edit the text back
- Adapts instantly when requirements change
⚠️ Fine-tuning (last resort)
- Needs curated data + training compute
- Hours-to-days per iteration
- Can narrow general capability
- Locks behaviour into a checkpoint you must maintain
- Worth it only when prompting genuinely can't reach the behaviour
EXAM TRAPS
PRACTICE
Q1 · Why is prompt engineering usually the first lever to pull?
Because it's the fastest and cheapest way to change behaviour: you edit text and get a new result in seconds, with no training data, no compute cost, and no risk to the model's general capabilities. It's also fully reversible — just change the words back.
Q2 · Your only problem is that responses take too long. Is a better prompt the fix?
No. Latency (and cost) are properties of the model, not the prompt. Anthropic explicitly notes these "can sometimes be more easily improved by selecting a different model." Change the model tier, don't rewrite the prompt.
Q3 · What three things does Anthropic assume you have before prompt engineering?
(1) A clear definition of your success criteria, (2) a way to empirically test against those criteria, and (3) a first-draft prompt to improve. Without them you're tuning blind — set them up first.