D4 · Prompt Engineering › Cluster 8 · Reliability, Hallucination & Iteration

8.4 · Evaluating & Iterating (Success Criteria)

Measure, then improve

CONCEPT Define measurable success criteria first

You cannot improve a prompt you can't measure. Before iterating, define success criteria that are specific, measurable, achievable, and relevant. Vague goals like "good performance" or "safe outputs" give you nothing to test against; concrete targets do.

⚠️ Unmeasurable

  • "Good sentiment analysis"
  • "Safe outputs"
  • "Accurate answers"

✅ Measurable

  • "F1 ≥ 0.85 on 10,000 held-out tweets"
  • "< 0.1% of 10,000 outputs flagged toxic"
  • "95% of citations resolve to a real source span"

Criteria are usually multidimensional — task fidelity, consistency, relevance, tone, latency, and price often matter at once. State a target for each dimension that's relevant to your app (citation accuracy is critical for a medical tool, minor for a casual chatbot).

CONCEPT Build representative test cases, then iterate empirically

With criteria fixed, build an eval set. Three design principles from Anthropic:

Grading methodGood for
Exact / string match (code)Categorical tasks — classification, intent detection
Code-based (ROUGE-L, cosine similarity)Summaries, consistency across paraphrases
LLM-based (Likert / binary / ordinal)Subjective qualities — tone, empathy, PII presence
Human gradingHighest-fidelity but slowest; reserve for what can't be automated
🔑 For LLM-as-judge grading, use a different model to grade than the one being evaluated, and give the judge a concrete rubric (e.g. "rate 1–5 for empathy").

DIAGRAM The empirical prompt-engineering loop

flowchart TD SC["1. Define success criteria
(specific + measurable)"] --> TC["2. Build test cases
(representative + edge cases)"] TC --> EV["3. Build evals
(automate grading)"] EV --> RUN["4. Run prompt vs. eval suite"] RUN --> PASS{"Meets criteria?"} PASS -->|"no"| REF["🟠 Analyze failures,
refine prompt"] REF --> RUN PASS -->|"yes"| VAL["5. Validate on held-out set"] VAL --> SHIP["🟢 Ship + monitor"] classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; class SHIP ok; class REF gate;

CODE An automated grader over a test set

Exact-match grading for a classification task, run over the whole eval set to produce one measurable number you can track across prompt versions:

def exact_match(output, expected):
    return output.strip().lower() == expected.lower()

# eval_set: representative inputs + edge cases + gold labels
correct = 0
for case in eval_set:
    r = client.messages.create(model="claude-opus-4-8", max_tokens=16,
        messages=[{"role":"user","content": prompt.format(text=case["input"])}])
    out = next(b.text for b in r.content if b.type == "text")
    correct += exact_match(out, case["label"])

accuracy = correct / len(eval_set)   # the number you iterate against
print(f"accuracy = {accuracy:.3f}")   # compare across prompt versions
The point isn't the metric itself — it's that every prompt change now produces a number that moves, so iteration is evidence-driven instead of vibes-driven.

TRADE-OFFS Automated vs. human grading

Automated gradingHuman grading
Cost / speedCheap, runs on every changeSlow, expensive
Signal qualityLower per-case, but high volumeHighest fidelity per case
Best forCategorical + code-gradable tasks; regression suitesNuanced quality the rubric can't capture
Anthropic's guidance: prioritize volume over perfection — a large auto-graded suite catches more regressions than a few hand-graded cases. Reserve humans for what genuinely can't be automated.

EXAM TRAPS

Trap: Iterating on a prompt by eyeballing a few outputs. → Without measurable criteria and a test set, you're tuning on vibes. Define success criteria and evals first, then iterate against a number.
Trap: Building an eval set of only "happy path" inputs. → You must include edge cases — irrelevant data, overly long or harmful inputs, ambiguous cases — because that's where real failures live.
Trap: Insisting every case be hand-graded before you'll trust the eval. → Anthropic says prioritize volume over perfection: more auto-graded cases beat a few perfect ones.
Trap: Using the same model as both worker and judge without thought. → For LLM-based grading, prefer a different model to grade, and give it an explicit rubric.

PRACTICE

Q1 · Rewrite "the classifier should be safe" as a measurable success criterion.

Something specific and quantified, e.g. "Fewer than 0.1% of outputs across 10,000 trials are flagged for toxicity by our content filter." It's specific, measurable, achievable against a benchmark, and relevant to the task.

Q2 · Why deliberately include ambiguous and harmful inputs in your test set?

Because evals must be task-specific and mirror the real input distribution, which includes edge cases: irrelevant/nonexistent data, overly long inputs, harmful input, and cases where humans disagree. Testing only clean inputs hides the failures that actually occur in production.

Q3 · You have 2,000 test cases but only budget to grade a handful by hand. What does Anthropic's guidance say?

Prioritize volume over perfection. Structure the task for automated grading (exact match, code-graded, or LLM-graded) and run all 2,000 — a large suite with slightly noisier grading catches far more regressions than a few perfectly hand-graded cases.