cleandataengineering.aiAgent Harness, drawn simply
You are here: Tech stack › L7 · Policy enforcement

Agent Harness Tutorial · Tech stack

L7 · Policy enforcement

Guardrails on the way in and on the way out. Family: Control · one turn, in order.

AZURE OPTIONSAI Content Safety✓ used: Prompt Shields + harm, fails openPrompt Shields on tool results◐ planned: F-124APIM llm-content-safety policy✗ dropped: two send-requests insteadGroundedness detection◐ planned: phase 2OUR CHOICEOur positioned rulesenforce() + evaluators.yaml, 30 checksContent Safety via APIMprompt and answerAPIM allow-list + rate limit30 calls a minuteOPEN-SOURCE OPTIONSLlama Prompt Guard 2◐ planned: small classifier, F-132Presidio + spaCy· considered: to decideLLM Guard✗ dropped: mostly droppedLlama Guard✗ dropped: an LLM, too slowNeMo Guardrails✗ dropped: wants to own the loopGuardrails AI✗ dropped: overlaps LLM GuardWHO DOES THE WORKAzure gives 35% · we code once 40% · each agent team still does 25%
Dark boxes are what we run. Solid boxes are used or planned; dashed boxes were considered or dropped (crossed). The bar: grey is what Azure gives, dark is what we code once, hatched is what each agent team writes.
Why our choiceWe own every position and every rule; every check that protects money is ours. Vendors supply detectors behind one port, switched on by one YAML line.
OptionWhere fromVerdictWhy
Our positioned rulesours✓ chosenenforce() + evaluators.yaml, 30 checks
Content Safety via APIMours✓ chosenprompt and answer
APIM allow-list + rate limitours✓ chosen30 calls a minute
AI Content SafetyAzure✓ usedPrompt Shields + harm, fails open
Prompt Shields on tool resultsAzure◐ plannedF-124
APIM llm-content-safety policyAzure✗ droppedtwo send-requests instead
Groundedness detectionAzure◐ plannedphase 2
Llama Prompt Guard 2open source◐ plannedsmall classifier, F-132
Presidio + spaCyopen source· consideredto decide
LLM Guardopen source✗ droppedmostly dropped
Llama Guardopen source✗ droppedan LLM, too slow
NeMo Guardrailsopen source✗ droppedwants to own the loop
Guardrails AIopen source✗ droppedoverlaps LLM Guard

How it works inside the harness: Checks.

Compared: Who provides each guardrail check?

Our rules (h…ourschosenAzure Conten…AzurechosenLlama Prompt…open sourceto buildPresidioopen sourceto decideLLM Guardopen sourcerejectedLlama Guardopen sourcerejectedNeMo Guardra…open sourcerejectedGuardrails AIopen sourcerejected
Dark: what we chose. Crossed and dashed: rejected or not for us. From the Series.
OptionFromVerdictKindWhere it runsWhat it catchesWhy
Our rules (harness + agent)ourschosenrules in codeevery position, before to reply30 checks, all builtevery check that protects money
Azure Content Safety (via APIM)Azurechosenservicegateway: prompt and replyPrompt Shields, harm categoriesallow-list, rate limit, harms
Llama Prompt Guard 2open sourceto buildsmall classifier modelpre_model and tool resultsprompt injectionfast; covers tool results (F-132)
Presidioopen sourceto decideclassic ML (spaCy)before the modelpersonal data to maskthe model may need identifiers
LLM Guardopen sourcerejectedscanner librarynot placedinvisible text, secrets, topicsnormalised() and router cover it
Llama Guardopen sourcerejectedLLM safety classifiernot placedharmful contenttoo slow for every turn
NeMo Guardrailsopen sourcerejecteddialogue rails frameworknot placedtopic and dialogue railswants to own the loop
Guardrails AIopen sourcerejectedvalidator frameworknot placedoutput validatorsoverlaps LLM Guard
DecisionOur own positioned rules make every check that protects money, and Azure Content Safety at the gateway adds Prompt Shields and harm screening. Prompt Guard 2 is the next addition for tool results; LLM Guard's Language scanner comes in only if we add Hindi replies.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

positions:
  # Inline, before the model is paid for: a fail ends the turn with the safe
  # reply, the refusal the loop already gives at PRE_MODEL (A11).
  pre_model:
    - {use: no_known_injection, on_fail: safe_reply}

  # Inline, on each tool result before it enters context: the result still goes
  # in, fenced as data like every result; any write later in the same run is
  # refused until the policyholder confirms it in a message of their own (A11).
  post_tool:
    - {use: no_instructions_in_result, on_fail: hold_writes}

  # Inline, before the policyholder reads it: a fail is the safe reply. Each runs
  # on the model's reply; those that need nothing but the reply also run on every
  # route's (a template's), where there are no tool results to ground against.
  reply:
    - {use: no_unclaimed_effect, on_fail: safe_reply}
    - {use: no_promise_of_cover, on_fail: safe_reply}
    - {use: no_ungrounded_entity, on_fail: safe_reply}
    - {use: no_superseded_state, on_fail: safe_reply}
    - {use: no_superseded_claim_state, on_fail: safe_reply}
    - {use: no_pii_echo, on_fail: safe_reply}
    - {use: money_in_rupees, on_fail: safe_reply}

  # After the turn, on what the policyholder was actually sent: a fail is a finding.
  online:
// WHAT IT IS
//   Azure AI Content Safety: a service that scores text for hate, sexual,
//   violence and self-harm content (0-7 severity), and "Prompt Shields", which
//   looks for a user trying to override the model's instructions.
//
// WHICH CONCERN IT SERVES
//   The Azure-provided inline check of Phase 1 (PLAN.html, "Azure-provided
//   inline checks"): with Groq behind APIM there are no Foundry guardrails, so
//   APIM sends each prompt and each completion here (infra/policies/
//   groq-api.xml) and refuses the call on a high score. Its verdicts are
//   logged for the guardrail_log plug-in (Tier 4).
//
// REGION
//   South India, not Central India: Content Safety's region table lists
//   southindia (content harms and prompt shields) and no other Indian region
//   (learn.microsoft.com/azure/ai-services/content-safety/region-availability,
//   updated 18 Sep 2026, read 9 Oct 2026). Text crosses to South India and back.
//
// AUTH
//   Keys disabled (disableLocalAuth): APIM calls it with its own managed
//   identity, which holds "Cognitive Services User" here.
//
// EXPECTED DEV COST
//   $0 on F0: 5,000 text records a month, 5 requests a second
//   (azure.microsoft.com/pricing/details/cognitive-services/content-safety and
//   the Content Safety overview's rate table, read 9 Oct 2026). Each model call
//   is two to three records (prompt shield, prompt, completion); past 5,000 a
//   month F0 refuses (429) and APIM's policy lets the call through unscreened,
//   flagged on the trace. Azure allows a limited number of free (F0) accounts
//   of a kind per subscription; if one exists already, provisioning fails here.