cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 3 · After the model › no_pii_echo

Agent Harness Tutorial · End to end · Zoom level 3

How we take care of it: no_pii_echo

no_pii_echo catches personal data repeated back.

Open: 11before loopopen ›Open: 22before modelopen ›Open: 33after modelopen ›Open: 44before toolopen ›Open: 55the toolopen ›Open: 66after toolopen ›Open: 77after loopopen ›Open: 88before replyopen ›It readsthe reply textno_pii_echorule · agentpassCarries onfailIf it failssafe replyWHERE IN THE CODE: F policy rules · evaluators.yaml reply
The strip at the top shows every check point where this check runs, in dark.

How we took care of it.

  1. Look for personal-data patterns: Aadhaar, PAN, phone, email, bank account.
  2. Runs after the model and before every reply.
It readsthe reply text
Kindrule
Provided byagent
If it failssafe reply
Where in the codeF policy rules · evaluators.yaml reply
Runs at3 · After the model, 8 · Before the reply
Real exampleTelemetry has its own redact() too, so the trace never holds them either.

More on this: The trace.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

def no_pii_echo(ctx: Context) -> Verdict:
    """Never read a bank account, driving licence, Aadhaar or PAN number back
    (AOAS `personal_data_in_conversation`): it puts it in a transcript, a log and
    a screenshot. Aadhaar and PAN are the harness's patterns (A12), the same
    ones redaction masks; a spaced Aadhaar number escaped `DIGITS` before."""
    if DIGITS.search(ctx.text) or LICENCE.search(ctx.text):
        return block("no_pii_echo", "an account- or licence-shaped number appeared in the reply")
    if aadhaar_numbers(ctx.text) or PAN.search(ctx.text):
        return block("no_pii_echo", "an Aadhaar or PAN number appeared in the reply")
    return ALLOW