Agent Harness Tutorial · End to end · Zoom level 1
Checks at every point
A check is a small piece of code that looks at one thing (a message, an answer, a tool request, a result) and says pass or block. The harness has checks at eight points along a turn, because each kind of mistake can only be seen at one place.
Almost every check is a plain rule. Rules are instant, free and give the same answer every time, so they can block safely in the middle of a turn. Azure adds three: a model allow-list, a rate limit, and Content Safety. No LLM judge runs inside a turn; judges are slow and costly, so they belong in offline eval.
What a block does depends on where it happens. Before or after the model, or before the reply, the turn ends with a safe reply: "I am not able to confirm that. Let me pass you to a colleague who can help." Before a tool, only that one call is refused, and the model is told why so it can choose again.
Fail closed. If a check itself crashes, that counts as a block. A broken guard must never mean an open door.
Want every detail? The long version is in the Series: policy at every position, with the live file.
Code view: input, process, output, the call stack, the classes
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
agent_harness/policy/__init__.py:63enforce(): run the rules for one point; the first block winsdef enforce(ctx: Context, rules: Sequence[Rule] | None = None) -> Verdict:
"""Run the rules for this position. First block wins.
A rule that raises **blocks**. That is the whole of AAC-0091: a guardrail
that errors open is believed and absent at once, and nobody goes looking for
a control they think they have.
"""
applicable = default_rules().get(ctx.position, ()) if rules is None else rules
with tel.span("agent.policy", **{"agent.policy.position": ctx.position.value}) as span:
for rule in applicable:
name = getattr(rule, "__name__", repr(rule))
try:
verdict = rule(ctx)
except Exception as exc: # noqa: BLE001 — deliberate: fail closed
span.set_attribute("agent.policy.blocked_by", name)
span.set_attribute("agent.policy.errored", True)
return block(name, f"rule failed and traffic was blocked: {exc}")
if verdict.blocked:
span.set_attribute("agent.policy.blocked_by", verdict.rule or name)
return verdict
return ALLOWagent_harness/policy/__init__.py:94What Rohan gets when a check blocksSAFE_REPLY = "I am not able to confirm that. Let me pass you to a colleague who can help."
claims_fnol/policy/__init__.py:246Which rules run at which point, for this agentDEFAULT_RULES: dict[Position, tuple[Rule, ...]] = {
# A11: known injection phrasing in the policyholder's words, before the model.
Position.PRE_MODEL: PLAN.at(Position.PRE_MODEL),
Position.POST_MODEL: OUTPUT_RULES,
# P-OWN-WORDS first; then A11's hold: no write after a result carrying instructions.
Position.PRE_TOOL: (policyholder_asked, *PLAN.at(Position.PRE_TOOL)),
Position.POST_TOOL: PLAN.at(Position.POST_TOOL),
Position.REPLY: REPLY_RULES,
}Tech choices (what we chose, Azure and open-source options): L7 · Policy enforcement