Agent Harness Tutorial · End to end · Zoom level 3
How we take care of it: no_promise_of_cover
no_promise_of_cover catches "your claim will be approved".
How we took care of it.
- Look for promises of cover, approval, an amount or a date: "will be approved", "is covered", "you will receive".
- Runs after the model, and again before every reply, templates included.
| It reads | the reply text |
| Kind | rule |
| Provided by | agent |
| If it fails | safe reply |
| Where in the code | F policy rules · evaluators.yaml reply |
| Runs at | 3 · After the model, 8 · Before the reply |
Real exampleThe system prompt forbids it too, but the rule makes sure.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
def no_promise_of_cover(ctx: Context) -> Verdict:
"""P-NO-PROMISE, R-COVERAGE at the output: never say or imply a claim is
covered, will be approved or paid, or for what amount. A sentence explaining
that only the assessment decides ("I cannot say whether it is covered") is
not a promise and is let through (F-004)."""
for sentence in re.split(r"(?<=[.!?])\s+", ctx.text):
if PROMISE.search(sentence) and not _EXPLAINING.search(sentence):
return block("no_promise_of_cover", f"promised an outcome: {sentence[:80]!r}")
return ALLOW