cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 3 · After the model › no_ungrounded_entity

Agent Harness Tutorial · End to end · Zoom level 3

How we take care of it: no_ungrounded_entity

no_ungrounded_entity catches a claim number no tool returned.

Open: 11before loopopen ›Open: 22before modelopen ›Open: 33after modelopen ›Open: 44before toolopen ›Open: 55the toolopen ›Open: 66after toolopen ›Open: 77after loopopen ›Open: 88before replyopen ›It readsthe reply, and the ids that toolsreturned this turnno_ungrounded_entityrule · agentpassCarries onfailIf it failssafe replyWHERE IN THE CODE: F policy rules · evaluators.yaml reply
The strip at the top shows every check point where this check runs, in dark.

How we took care of it.

  1. Find every CLM- and POL- number in the reply.
  2. Each must have come back from a tool this turn, or from a fresh re-read.
It readsthe reply, and the ids that tools returned this turn
Kindrule
Provided byagent
If it failssafe reply
Where in the codeF policy rules · evaluators.yaml reply
Runs at3 · After the model
Real exampleStops a made-up claim number from ever reaching Rohan.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

def no_ungrounded_entity(ctx: Context) -> Verdict:
    """Every reference, amount and date in the reply appears in what the tools
    returned this turn — so no figure nobody decided reaches the policyholder."""
    if not ctx.tool_results:
        return ALLOW
    evidence = " ".join(f"{r.structured} {r.text}" for r in ctx.tool_results).lower()
    for found in set(IDENTIFIER.findall(ctx.text)):
        if found.lower() not in evidence:
            return block("no_ungrounded_entity", f"cited {found!r}, which no tool returned")
    for date in ISO_DATE.findall(ctx.text):
        if date not in evidence:
            return block("no_ungrounded_entity", f"stated the date {date!r}, unsupported")
    flat = _flat(evidence)
    for match in MONEY.finditer(IDENTIFIER.sub(" ", ctx.text)):
        amount = match.group(1) or match.group(2)
        if amount and (_flat(amount).lstrip("0") or "0") not in flat:
            return block("no_ungrounded_entity", f"stated the figure {amount!r}, unsupported")
    return ALLOW