cleandataengineering.aiEval Harness, drawn simply
You are here: Five patterns

Eval Harness Tutorial · Five patterns

Five patterns for whole conversations

Most of the questions ops teams ask about an agent fit five shapes. Each shape is a pattern: plain code that reads a session, written once by us. A domain team fills it in with their own tool names and limits in YAML; they don't write code.

PatternThe kind of question it answers
requires_beforeWas the claim read before it was paid?
argument_limitWas a refund above ₹10,000 sent?
claim_needs_actionDid it say “paid” when nothing was paid?
decision_vs_shouldWas a person asked to approve a large payout?
consistencyDid it state an amount no system gave it?
How every pattern answers

Each pattern counts how many things it checked and which failed. The score is the share that held (1.0 when nothing applied). Any failure fails the check, and each failure says what and at which step. No AI model is involved: the same session always gets the same verdict.

Each page below shows one passing and one failing conversation. Both were run through the real code when this page was built; the verdicts are its actual output.

Look inside