D4 · Prompt Engineering › Cluster 1 · Foundations & Being Clear

1.1 · What Prompt Engineering Is & When to Use It

Prompt before you fine-tune

CONCEPT Prompt engineering is the fastest, cheapest lever

Prompt engineering is the practice of shaping a model's behaviour by editing the words you send it — the instructions, context, examples, and structure of the prompt — rather than changing the model's weights. For most use cases it is the first lever you should reach for, because it is far faster and cheaper to iterate on than the alternatives.

Set against fine-tuning, the contrast is stark. A prompt edit gives you a new result in seconds; a fine-tune requires curating a training dataset, running a training job, and evaluating a new checkpoint — hours to days and real compute cost per iteration. Prompting also preserves the model's general capabilities (fine-tuning can narrow them), needs no labelled data, and adapts instantly when requirements change. That is why the standard guidance is: exhaust prompt engineering first, and only move to fine-tuning when prompting genuinely can't close the gap.

🔑 Mental model: prompting steers a capable model you already have; fine-tuning bakes a new model. Steering is cheap, reversible, and instantaneous — start there.
LeverIteration speedCostReach for it when…
Prompt engineeringSeconds~None (just tokens)Almost always — the default first move
Few-shot examples in promptMinutesExtra input tokensFormat/tone/structure need pinning down
Model selectionMinutesPer-token price deltaLatency or cost is the real problem, not quality
Fine-tuningHours–daysData + training computePrompting truly can't reach the behaviour, and you have data

CONCEPT When prompt engineering is not the answer

Prompting is powerful but not universal. Anthropic's own guidance is explicit: "Not every success criteria or failing eval is best solved by prompt engineering. For example, latency and cost can sometimes be more easily improved by selecting a different model." A prompt cannot make a model faster or cheaper — those are properties of the model tier you choose. If your failing metric is speed or price, the fix is a smaller/faster model (or fast mode), not a cleverer prompt.

Likewise, before you even start tuning, Anthropic assumes you have three things in place. Missing any of them means you're optimizing blind:

Prompt engineering assumes you already have: (1) a clear definition of success criteria, (2) a way to empirically test against those criteria, and (3) a first-draft prompt to improve. Without these, establish them first (that's subtopic 1.2).
Right tool per problem: quality & behaviour → prompt engineering; latency & cost → model choice; a durable capability prompting can't reach → fine-tuning.

DIAGRAM Which lever fixes this failing eval?

flowchart TD A["Failing eval / gap in behaviour"] --> Q{"What kind of gap?"} Q -->|"Quality, format, tone, following instructions"| P["🟢 Prompt engineering
edit the prompt — seconds"] Q -->|"Too slow / too expensive"| M["🔵 Change the model tier
not the prompt"] Q -->|"Durable capability prompting can't reach"| F{"Have data + eval budget?"} F -->|"yes"| FT["🟠 Fine-tune
hours–days, real cost"] F -->|"no"| P classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef alt fill:#cfe2ff,stroke:#0d6efd,color:#084298; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class P ok; class M alt; class FT gate;

CODE The same behaviour: prompt tweak vs. the fine-tune you avoided

A behaviour that looks like it needs training — "always answer as a terse SRE" — is usually one prompt away. This is the whole argument for reaching for prompting first: no dataset, no training run, instant result.

# ❌ The instinct: "the model won't behave, let's fine-tune."
#    → curate 1000s of examples, run a training job, eval a new checkpoint.
#    → hours-to-days, real compute cost, PER iteration.

# ✅ The lever to try FIRST: a system prompt. New behaviour in seconds.
resp = client.messages.create(
    model="claude-opus-4-8",
    max_tokens=1024,
    system=(
        "You are a senior SRE. Answer in at most 3 sentences. "
        "Lead with the action to take, then the one-line reason. "
        "No preamble, no pleasantries."
    ),
    messages=[{"role": "user", "content": "Disk is 95% full on prod-db-1. What now?"}],
)
# Edit the system string → re-run → new behaviour. That loop is the point.

TRADE-OFFS Prompt engineering vs. fine-tuning

✅ Prompt engineering (start here)

  • Iterate in seconds; no training pipeline
  • No labelled dataset required
  • Preserves the model's general capabilities
  • Reversible — just edit the text back
  • Adapts instantly when requirements change

⚠️ Fine-tuning (last resort)

  • Needs curated data + training compute
  • Hours-to-days per iteration
  • Can narrow general capability
  • Locks behaviour into a checkpoint you must maintain
  • Worth it only when prompting genuinely can't reach the behaviour
🔑 And remember the third door: if the failing metric is latency or cost, neither prompting nor fine-tuning is the fix — change the model.

EXAM TRAPS

Trap: "The model is too slow, let's engineer a better prompt." → A prompt can't change inference speed. Latency and cost are model-tier problems — pick a faster/cheaper model.
Trap: "Reach for fine-tuning to change behaviour." → Fine-tuning is the last resort. Try prompt engineering first — it's faster, cheaper, needs no data, and is reversible.
Trap: Starting to tune with no success criteria or eval. → Anthropic assumes you have clear success criteria + an empirical test + a draft prompt before prompt engineering. Establish those first.
Trap: Thinking prompting and fine-tuning are interchangeable knobs. → They aren't. Prompting steers a fixed model; fine-tuning bakes a new one. Different cost, speed, and reversibility.

PRACTICE

Q1 · Why is prompt engineering usually the first lever to pull?

Because it's the fastest and cheapest way to change behaviour: you edit text and get a new result in seconds, with no training data, no compute cost, and no risk to the model's general capabilities. It's also fully reversible — just change the words back.

Q2 · Your only problem is that responses take too long. Is a better prompt the fix?

No. Latency (and cost) are properties of the model, not the prompt. Anthropic explicitly notes these "can sometimes be more easily improved by selecting a different model." Change the model tier, don't rewrite the prompt.

Q3 · What three things does Anthropic assume you have before prompt engineering?

(1) A clear definition of your success criteria, (2) a way to empirically test against those criteria, and (3) a first-draft prompt to improve. Without them you're tuning blind — set them up first.