Agent Harness Tutorial · End to end · Zoom level 2
How policies are implemented
A policy starts as one sentence a business owner agrees to, such as "a reply never says or implies that a claim is covered". It ends as a check that blocks a reply in 0.04 ms, a line in the trace when it fires, and a test that fails the release if anyone breaks it. Every step carries the same id, P-NO-PROMISE, so you can follow it end to end.
Two shapes of policy. Some policies are about what may be said: those become rules that read the reply (left). Others are about who may do what: those become authority, built into which tools exist, a limit in configuration, a workflow that waits for a person, and a check at the far end (right). Neither is left to the model's judgement; the system prompt says the same things, but only as a first line of defence.
| Policy (spec id) | In words | How it is implemented | Where |
|---|---|---|---|
| P-NO-PROMISE | never say a claim is covered or will be paid | reply rules no_promise_of_cover, no_ungrounded_entity; router refusal R-COVERAGE | check point 3, 8, router |
| P-PAYOUT | over ₹25,000 needs a claims handler | request_payout + DBOS approval + covers() at the claims system | payout flow |
| P-APPROVER | never approved by the policyholder whose claim it is | covers(): decided_by must differ from the holder | approval covers it |
| P-OWN-WORDS | act only on what the policyholder asked for | consent from his own words + policyholder_asked before every write | policyholder_asked |
| P-OWNERSHIP | only your own policies and claims | holder from the token; every query filtered by it | only his rows |
| P-WITHDRAW | withdraw until assessment starts | the claims system's precondition on withdraw_claim | tool call |
To add a policy: write the sentence in the spec with an id; write the rule once in RULES (or the authority in the spec and config); add one line to evaluators.yaml; write a test marked with the policy id. The app refuses to start if the YAML places a rule where its inputs never exist, or one slower than 50 ms at an inline point.
To change a policy: a limit lives in App Configuration (no deploy); a placement or an on-fail action lives in the YAML; only a new kind of rule needs code.
Code view: input, process, output, the call stack, the classes
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
clean-ai-engineering/drafts/examples/motor-claims-fnol.aoas.yaml:391Step 1: the policy, in the spec P-NO-PROMISE:
rule: a reply never says or implies that a claim is covered, will be approved or paid, or for what amount, unless the claim's own status and approved amount say so
via: [get_claim]
concern: safety
facet: operational constraintclaims_fnol/policy/__init__.py:91Step 2: the ruledef no_promise_of_cover(ctx: Context) -> Verdict:
"""P-NO-PROMISE, R-COVERAGE at the output: never say or imply a claim is
covered, will be approved or paid, or for what amount. A sentence explaining
that only the assessment decides ("I cannot say whether it is covered") is
not a promise and is let through (F-004)."""
for sentence in re.split(r"(?<=[.!?])\s+", ctx.text):
if PROMISE.search(sentence) and not _EXPLAINING.search(sentence):
return block("no_promise_of_cover", f"promised an outcome: {sentence[:80]!r}")
return ALLOWclaims_fnol/policy/__init__.py:190Step 3: registered, with what each rule needsRULES: dict[str, RuleSpec] = {
"no_unclaimed_effect": RuleSpec(no_unclaimed_effect, GROUNDED),
"no_promise_of_cover": RuleSpec(no_promise_of_cover),
"no_ungrounded_entity": RuleSpec(no_ungrounded_entity, GROUNDED),
"no_superseded_state": RuleSpec(no_superseded_state, GROUNDED),
"no_superseded_claim_state": RuleSpec(no_superseded_claim_state, GROUNDED),
"no_pii_echo": RuleSpec(no_pii_echo),
"money_in_rupees": RuleSpec(money_in_rupees),
}claims_fnol/policy/evaluators.yaml:45Step 4: placed at the reply point, with what a fail does reply:
- {use: no_unclaimed_effect, on_fail: safe_reply}
- {use: no_promise_of_cover, on_fail: safe_reply}
- {use: no_ungrounded_entity, on_fail: safe_reply}
- {use: no_superseded_state, on_fail: safe_reply}
- {use: no_superseded_claim_state, on_fail: safe_reply}
- {use: no_pii_echo, on_fail: safe_reply}
- {use: money_in_rupees, on_fail: safe_reply}agent_harness/policy/__init__.py:63Step 5: enforced; the first block wins, a crash blocksdef enforce(ctx: Context, rules: Sequence[Rule] | None = None) -> Verdict:
"""Run the rules for this position. First block wins.
A rule that raises **blocks**. That is the whole of AAC-0091: a guardrail
that errors open is believed and absent at once, and nobody goes looking for
a control they think they have.
"""
applicable = default_rules().get(ctx.position, ()) if rules is None else rules
with tel.span("agent.policy", **{"agent.policy.position": ctx.position.value}) as span:
for rule in applicable:
name = getattr(rule, "__name__", repr(rule))
try:
verdict = rule(ctx)
except Exception as exc: # noqa: BLE001 — deliberate: fail closed
span.set_attribute("agent.policy.blocked_by", name)
span.set_attribute("agent.policy.errored", True)
return block(name, f"rule failed and traffic was blocked: {exc}")
if verdict.blocked:
span.set_attribute("agent.policy.blocked_by", verdict.rule or name)
return verdict
return ALLOWtests/test_policy.py:152Step 7: a test that names the policy it proves@pytest.mark.discharges("P-NO-PROMISE", "AHC-0094")
@pytest.mark.parametrize(
("why", "reply", "results", "blocked"), GROUNDING, ids=[g[0] for g in GROUNDING]
)
def test_every_reference_and_figure_is_grounded(
why: str, reply: str, results: tuple[ToolResult, ...], blocked: bool
) -> None:
assert no_ungrounded_entity(said(reply, *results)).blocked is blocked
SUPERSEDED = [
(clean-ai-engineering/drafts/examples/motor-claims-fnol.aoas.yaml:368The authority-shaped policy: P-PAYOUT's limit, in the spec authority:
agent_when:
# ASSUMPTION: approved glass claims and small amounts are paid at once;
# anything larger needs a claims handler. ₹25,000 is a placeholder until
# Q-AUTO-LIMIT's measurement sets it.
- {field: approved_amount, at_most: 25000}
otherwise: human_approvalclaims_system/server.py:187And checked again at the far end before any money movesdef covers(
approval: ApprovalRecord | None,
*,
holder: str,
claim: Row,
key: str | None,
now: float,
limit: Decimal = AOAS_LIMIT,
) -> str | None:
"""Why this approval record does not cover paying this claim now, or `None`.
The call is rebuilt as this system received it — the claim, the amount this
system holds for it, the policyholder it verified, the key on the call — and
its digest must be the record's (`agent_harness.contracts.records.refusals`).
`limit` is the automatic limit read for this payout."""
if approval is None:
return "issue_payout needs an approval, and none this system can read was named"
amount = Decimal(str(claim.get("approved_amount", 0)))
failed = refusals(
approval,
action="issue_payout",
args={"claim_id": str(claim["id"]), "amount": str(amount)},
requested_for=holder,
idempotency_key=key or "",
now=now,
)
if approval.decided_by in (None, holder):
failed.append("nobody but the policyholder approved it")
if amount > limit and approval.decided_by == AUTOMATIC_APPROVER:
failed.append(f"{amount} is above the {limit} automatic limit, and no person decided it")
if not failed:
return None
return f"approval {approval.id} does not cover this payout: {'; '.join(failed)}"