cleandataengineering.aiAgent Harness, drawn simply
You are here: Tech stack › PV · Privacy & data protection

Agent Harness Tutorial · Tech stack

PV · Privacy & data protection

Personal data used only for this case, kept in India, masked, and forgotten on request. Family: Control · one turn, in order.

AZURE OPTIONSAI Language (PII)◐ planned: in the design, not deployedEncryption at rest, private endpoints◐ planned: in the designOUR CHOICEredact() in telemetryAadhaar, PAN, licence, accountsno_pii_echo ruleon every replyIndia regionsCentral + South Indiaforget() + 30-day retentionbuilt, not scheduledOPEN-SOURCE OPTIONSPresidio· considered: to decide: the model needs some idsWHO DOES THE WORKAzure gives 45% · we code once 40% · each agent team still does 15%
Dark boxes are what we run. Solid boxes are used or planned; dashed boxes were considered or dropped (crossed). The bar: grey is what Azure gives, dark is what we code once, hatched is what each agent team writes.
Why our choiceRedaction, entitlement and erasure are domain rules we code once. Azure supplies residency and encryption.
OptionWhere fromVerdictWhy
redact() in telemetryours✓ chosenAadhaar, PAN, licence, accounts
no_pii_echo ruleours✓ chosenon every reply
India regionsours✓ chosenCentral + South India
forget() + 30-day retentionours✓ chosenbuilt, not scheduled
AI Language (PII)Azure◐ plannedin the design, not deployed
Encryption at rest, private endpointsAzure◐ plannedin the design
Presidioopen source· consideredto decide: the model needs some ids

How it works inside the harness: The trace.

Compared: How is personal data found and masked?

Regex redact() (ours)ourschosenno_pii_echo reply ruleourschosenMicrosoft Presidioopen sourceto decideAzure AI Language PIIAzureplanned
Dark: what we chose. Crossed and dashed: rejected or not for us. Added here (not a Series table).
OptionFromVerdictWhere it runsIndia IDs (Aadhaar, PAN)Network callHandles free textStatus
Regex redact() (ours)ourschosenevery payload and log lineyes, Aadhaar Verhoeff-checkednoneknown patterns onlybuilt
no_pii_echo reply ruleourschosenpost_model and replyyesnoneknown patterns onlybuilt
Microsoft Presidioopen sourceto decidein our process (spaCy)built-in IN_PAN, IN_AADHAARnoneyes: names, placesto decide: model needs IDs
Azure AI Language PIIAzureplannedAzure serviceyes, India entity typesyesyesdesign: mask before model
DecisionOur regex redact() and the no_pii_echo rule are built and cover the identifiers that matter, on every log line and every reply. Masking before the model, with Presidio or Azure AI Language, is still to decide because the model sometimes needs the identifiers.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

def redact(text: str, *, limit: int = 4000) -> str:
    """The single redaction point. Everything captured onto a span comes here.

    Deliberately crude: deterministic regexes, no model call, no network. A
    detector that can be wrong slowly is worse than one that is obviously
    approximate — this is a floor, not a compliance control.
    """
    for pattern, replacement in (*_LIBRARY, *_registered):
        text = pattern.sub(replacement, text)
    if len(text) > limit:
        return text[:limit] + f"…[truncated {len(text) - limit} chars]"
    return text