Agent Harness Tutorial · End to end · Zoom level 3
How we take care of it: Content Safety on the answer
Content Safety on the answer catches harmful text.
How we took care of it.
- The same harm check as on the prompt, run on the completion.
- A harmful answer becomes a 403 before it reaches the harness.
| It reads | the model's answer, on the way back through APIM |
| Kind | classic ML |
| Provided by | Azure APIM |
| If it fails | 403 → "I am having trouble answering right now." |
| Where in the code | infra/policies/groq-api.xml |
| Runs at | 3 · After the model |
Real exampleNo real answer has been blocked so far.
More on this: The APIM gateway.
How it is done: the real code
Cut from the live repositories each time the site is built, so it always matches what runs.
<set-variable name="completion" value="@{
if (context.Response.StatusCode != 200) { return ""; }
var body = context.Response.Body.As<JObject>(preserveContent: true);
var text = (string)body["choices"]?[0]?["message"]?["content"] ?? "";
return text.Length > 9000 ? text.Substring(0, 9000) : text;
}" />
<choose>
<when condition="@(((string)context.Variables["completion"]).Length > 0)">
<send-request mode="new" response-variable-name="completionHarm" timeout="10" ignore-error="true">
<set-url>{{content-safety-endpoint}}contentsafety/text:analyze?api-version=2024-09-01</set-url>
<set-method>POST</set-method>
<set-header name="Content-Type" exists-action="override"><value>application/json</value></set-header>
<set-body>@(new JObject(new JProperty("text", (string)context.Variables["completion"]), new JProperty("outputType", "FourSeverityLevels")).ToString())</set-body>
<authentication-managed-identity resource="https://cognitiveservices.azure.com" client-id="{{apim-identity-client-id}}" />
</send-request>
<set-variable name="completionVerdict" value="@{
var harm = context.Variables.GetValueOrDefault<IResponse>("completionHarm");
if (harm == null || harm.StatusCode != 200) { return "unavailable"; }
foreach (var c in (JArray)harm.Body.As<JObject>(preserveContent: true)["categoriesAnalysis"]) {
if ((int)c["severity"] >= {{content-safety-threshold}}) { return "block:" + (string)c["category"]; }
}
return "pass";
}" />
<choose>
<when condition="@(((string)context.Variables["completionVerdict"]).StartsWith("block"))">
<return-response>
<set-status code="403" reason="Blocked by Content Safety" />
<set-header name="x-content-safety" exists-action="override"><value>@((string)context.Variables["completionVerdict"] + "; prompt=" + context.Variables.GetValueOrDefault<string>("promptVerdict", "skipped") + "; completion=" + (string)context.Variables["completionVerdict"])</value></set-header>
<set-header name="Content-Type" exists-action="override"><value>application/json</value></set-header>
<set-body>{"error":{"code":"content_safety","message":"the completion was blocked by Content Safety"}}</set-body>