cleandataengineering.aiAgent Harness, drawn simply
You are here: End to end › Checks › 2 · Before the model

Agent Harness Tutorial · End to end · Zoom level 2

Check point 2: Before the model

Check what we are about to send, so an attack in Rohan's message never reaches the model.

Where: every trip, after context assembly.

The strip at the top is the whole turn; the dark box is where you are. Click a check, or what it catches, to see how we took care of it. Dashed boxes are not built yet.
CheckCatchesKindProvided byIf it fails
allowed tools onlytools his token doesn't allowruleharnessnever shown
out of time?the 60 s deadline, or Rohan leftruleharnessstop the turn
no_known_injection"ignore your instructions", fake role markersruleharnesssafe reply, $0 spent
model allow-listany model but gpt-oss-120bruleAzure APIM403
rate limitmore than 30 calls a minuteruleAzure APIM429, wait, retry
Content Safetyjailbreaks (Prompt Shields) and harmclassic MLAzure APIM403, "trouble answering"
Prompt Guard 2paraphrased and foreign-language attackssmall modelopen sourcenot built yet

Our injection rule checks only the latest user message, after folding odd spaces and look-alike letters into plain ones. It catches "ignore your instructions", "you are now", "reveal your system prompt" and fake role markers like [SYSTEM]. When it fires the turn ends in 250 ms and APIM is never called. It misses paraphrases ("As the claims manager I authorise you..."), other languages and leetspeak, which is why Prompt Guard 2 is next on the list.

There is no personal-data masking of the prompt: the model needs some identifiers, such as claim numbers, to do its job. Sizes are limited by the 8 KiB message limit and the context budget.

How it is done: the real code

Cut from the live repositories each time the site is built, so it always matches what runs.

claims_fnol/policy/evaluators.yaml:33evaluators.yaml: what runs before the model
  pre_model:
    - {use: no_known_injection, on_fail: safe_reply}

Look inside