cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 3 · After the model › no_promise_of_cover

Agent Harness Tutorial · End to end · Zoom level 3

How we take care of it: no_promise_of_cover

no_promise_of_cover catches "your claim will be approved".

Open: 11before loopopen ›Open: 22before modelopen ›Open: 33after modelopen ›Open: 44before toolopen ›Open: 55the toolopen ›Open: 66after toolopen ›Open: 77after loopopen ›Open: 88before replyopen ›It readsthe reply textno_promise_of_coverrule · agentpassCarries onfailIf it failssafe replyWHERE IN THE CODE: F policy rules · evaluators.yaml reply
The strip at the top shows every check point where this check runs, in dark.

How we took care of it.

  1. Look for promises of cover, approval, an amount or a date: "will be approved", "is covered", "you will receive".
  2. Runs after the model, and again before every reply, templates included.
It readsthe reply text
Kindrule
Provided byagent
If it failssafe reply
Where in the codeF policy rules · evaluators.yaml reply
Runs at3 · After the model, 8 · Before the reply
Real exampleThe system prompt forbids it too, but the rule makes sure.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

def no_promise_of_cover(ctx: Context) -> Verdict:
    """P-NO-PROMISE, R-COVERAGE at the output: never say or imply a claim is
    covered, will be approved or paid, or for what amount. A sentence explaining
    that only the assessment decides ("I cannot say whether it is covered") is
    not a promise and is let through (F-004)."""
    for sentence in re.split(r"(?<=[.!?])\s+", ctx.text):
        if PROMISE.search(sentence) and not _EXPLAINING.search(sentence):
            return block("no_promise_of_cover", f"promised an outcome: {sentence[:80]!r}")
    return ALLOW