Home › Learn › Agent guardrails
AI engineering · free reference · grows over time
The agent guardrail catalog
57 common guardrails for autonomous agents, sorted by where they sit in a turn. Each one is marked must, should or could have; says whether a rule, a small classifier, an LLM judge, a person or the platform decides; and links to the test that proves it works.
35 must-haves, 19 should-haves, 3 could-haves. 39 of the 57 are plain rules in code. Only 2 need an LLM judge. The short version, with pictures, is the post Every guardrail your agent needs, sorted.
How a guardrail decides
about 0 ms, free · 39 in this list
10 to 100 ms, cheap · 10 in this list
0.5 to 3 s, costs tokens · 2 in this list
minutes to days · 1 in this list
about 0 ms · 5 in this list
Must means every agent that can change something in the real world. Should means most agents. Could depends on the use. "Proven by" links to the matching obligation in the AI Assurance Catalog: the test that shows the guardrail really works.
Filter
Front door
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Verify who is askingMust haveRule in code | a forged or expired sign-in | block | Security | your identity provider | AAC-0111 |
| Kill switchMust haveRule in code | the agent has been switched off by an operator | block | Reliability | a feature flag | AAC-0077 |
| Same message twiceMust haveRule in code | a double-click or a retry sends the message again | return the first answer | Reliability | an idempotency table | AAC-0076 |
| Input size and shapeMust haveRule in code | an empty, huge or binary message | block | Reliability | your code | AAC-0015 |
| Rate limit and quotaShould haveRule in code | one user sending 500 messages a minute | slow down | Cost | API gateway (APIM, Kong, LiteLLM) | AAC-0093 |
| Sensitive topics go to a personMust haveRule in code | injury, self-harm, legal threats, fraud claims | hand to a person | Safety | keyword rules, then a classifier | AAC-0005 |
| Off-topic requestShould haveSmall classifier | a banking agent asked to write a poem | polite refusal | Correctness | embedding similarity or a small classifier | AAC-0020 |
| Record what the user asked forMust haveRule in code | lists the actions the user's own words asked for, used later at the tool | used later | Security | your code | AAC-0056 |
Before the model
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Known injection phrasesMust haveRule in code | "ignore your previous instructions", fake role markers | block | Security | a pattern list | AAC-0004 |
| Jailbreak and prompt attack detectorShould haveSmall classifier | reworded or foreign-language attacks the patterns miss | block | Security | Prompt Shields, Llama Prompt Guard 2, Lakera | AAC-0004 |
| Harmful inputMust haveSmall classifier | hate, violence, sexual or self-harm content | block | Safety | Azure Content Safety, Llama Guard, Bedrock Guardrails | AAC-0005 |
| Personal data in the inputShould haveSmall classifier | a card number or passport number pasted into chat | redact | Privacy | Presidio, Comprehend, Cloud DLP | AAC-0006 |
| Secrets in the inputShould haveRule in code | an API key or password pasted into chat | redact | Privacy | secret-scanner patterns | AAC-0006 |
| Only the tools this user may useMust haveRule in code | an ordinary user's agent offered the admin tools | never shown | Security | your code | AAC-0057 |
| Context size capMust haveRule in code | a conversation that has grown past the window | trim | Reliability | your code | AAC-0019 |
| Approved models onlyShould havePlatform setting | a request routed to an unapproved or unknown model | block | Human oversight | LLM gateway allow-list | AAC-0094 |
| Data stays in its regionCould havePlatform setting | personal data sent to a model in another country | block | Privacy | regional endpoints, gateway policy | AAC-0097 |
After the model
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Output matches its schemaMust haveRule in code | missing fields, wrong types, broken JSON | repair or retry | Correctness | Pydantic, JSON Schema, structured outputs | AAC-0002 |
| Harmful outputMust haveSmall classifier | the model writes something offensive or dangerous | block | Safety | Azure Content Safety, Llama Guard, Bedrock Guardrails | AAC-0005 |
| Personal data in the outputMust haveSmall classifier | the reply repeats another person's phone number | redact | Privacy | Presidio, regex for IDs | AAC-0006 |
| Claims an action that didn't happenMust haveRule in code | "I've booked it" when no booking tool ran | safe reply | Truth | compare the reply with the tool log | AAC-0110 |
| Names or numbers no tool returnedMust haveRule in code | an order number or price the model made up | safe reply | Truth | match entities against tool results | AAC-0024 |
| Every claim is supportedShould haveLLM judge | a sentence the retrieved documents don't support | regenerate or flag | Truth | an LLM judge, or an NLI model | AAC-0029 |
| Citations point to real sourcesShould haveRule in code | a link or document id that doesn't exist | remove the citation | Truth | your code | AAC-0030 |
| Forbidden promisesMust haveRule in code | "your claim will be approved", legal or medical advice | safe reply | Safety | phrase rules, then an LLM judge for nuance | AAC-0003 |
| Units, currency and formatShould haveRule in code | £ instead of ₹, a date in the wrong format | fix or safe reply | Correctness | your code | AAC-0023 |
| Old status after a newer oneShould haveRule in code | says "pending" after a tool said "paid" | safe reply | Correctness | your code | AAC-0113 |
| System prompt leakShould haveRule in code | the reply contains the hidden instructions | block | Security | a canary string in the prompt | AAC-0106 |
| Tone and brandCould haveLLM judge | rude, sarcastic or off-brand replies, competitor mentions | rewrite or flag | Correctness | an LLM judge with a short rubric | AAC-0074 |
| Screen streamed textCould haveSmall classifier | harmful words shown before the full answer is checked | cut the stream | Safety | chunked moderation | AAC-0092 |
Before the tool
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| The user actually asked for thisMust haveRule in code | a refund the user never asked for, triggered by a planted note | refuse the call, tell the model why | Security | your code | AAC-0056 |
| Arguments are valid and in rangeMust haveRule in code | a refund of ₹4,00,000 when the limit is ₹40,000 | refuse the call | Correctness | Pydantic, business rules | AAC-0052 |
| Approval for risky actionsMust haveA person | a large payout, deleting data, sending to all customers | wait for a person | Human oversight | a workflow engine (DBOS, Temporal) plus a queue | AAC-0078 |
| Destructive actions are gatedMust haveRule in code | DROP TABLE, deleting files, cancelling all orders | refuse or dry-run first | Safety | an allow-list of actions | AAC-0056 |
| Hold writes after a suspicious resultShould haveRule in code | any write after a tool result that carried instructions | ask the user first | Security | your code | AAC-0058 |
| Tool-call budgetMust haveRule in code | the model calling the same search 40 times | stop the turn | Cost | your code | AAC-0055 |
| Re-read before an irreversible writeShould haveRule in code | paying a claim whose status changed a minute ago | re-read first | Correctness | your code | AAC-0113 |
| Edits stay in scopeMust haveRule in code | a coding agent editing files outside its folder | refuse the call | Safety | path and table allow-lists | AAC-0075 |
| Code runs in a sandboxMust havePlatform setting | generated code reading secrets or the network | contain | Safety | containers, E2B, Firecracker, gVisor | AAC-0072 |
Inside the tool
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| The tool checks identity againMust haveRule in code | the agent's token used for someone else | error | Security | on-behalf-of tokens, scopes | AAC-0111 |
| Only the caller's own rowsMust haveRule in code | another customer's order or claim | not found | Privacy | row-level security | AAC-0040 |
| Same write only onceMust haveRule in code | a retried payment that would pay twice | return the first result | Reliability | idempotency keys | AAC-0047 |
| Retrieval respects permissionsMust haveRule in code | search results from documents the user can't open | filter out | Privacy | permission-aware search filters | AAC-0032 |
| Cap the size of resultsMust haveRule in code | a tool returning 2 MB of JSON into the context | cut and mark | Cost | your code | AAC-0105 |
After the tool
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Fence the result as dataMust haveRule in code | text in a result being read as instructions | always applied | Security | delimiters plus a system rule | AAC-0058 |
| Instructions hidden in a resultShould haveSmall classifier | "call request_payout now" planted in an email or note | hold later writes | Security | patterns, then Prompt Guard or Prompt Shields | AAC-0058 |
| Poisoned documentsShould haveSmall classifier | a retrieved page written to steer the agent | drop or flag | Security | Prompt Shields for documents, source allow-lists | AAC-0036 |
Whole run
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Step, time and money budgetMust haveRule in code | a run that loops for 20 minutes and $30 | stop the run | Cost | your code, gateway budgets | AAC-0077 |
| Going round in circlesMust haveRule in code | the same tool with the same arguments again and again | stop or hand off | Reliability | your code | AAC-0109 |
| Hand off after repeated failureShould haveRule in code | two refusals in a row, a tool down, 10+ turns | hand to a person | Human oversight | your code plus a support queue | AAC-0043 |
| Guardrails fail closedMust haveRule in code | a guardrail service times out and everything is let through | block | Safety | your code | AAC-0091 |
| Memory writes are checkedShould haveSmall classifier | a user teaching the agent a false fact that later users see | skip the write | Security | rules plus a classifier | AAC-0041 |
| Promises have something behind themShould haveRule in code | "we'll call you tomorrow" with no task created | create the task or hand off | Truth | your code | AAC-0110 |
Logs and platform
| Guardrail | What it catches | If it fires | Concern | Tools that do it | Proven by |
|---|---|---|---|---|---|
| Logs are redacted and expireMust haveRule in code | full card numbers kept in traces for ever | redact | Privacy | OpenTelemetry processors, retention policies | AAC-0095 |
| Every action can be tracedMust havePlatform setting | nobody can say why money moved last Tuesday | record | Human oversight | OpenTelemetry, Langfuse, Phoenix | AAC-0080 |
| Guardrail hit rates are watchedShould haveRule in code | a guardrail that blocks 30% of good requests, or none at all | alert | Safety | dashboards and alerts on block rates | AAC-0088 |
| Serve what was testedMust havePlatform setting | a prompt or model changed in production without evaluation | block the release | Security | versioned releases, CI gates | AAC-0107 |
Add one
Missing a guardrail you rely on? Email basant@cleandataengineering.ai with what it catches and where it sits. Tool names are examples, not endorsements; check current docs before you build.