CONCEPT Guardrails wrap the model on both sides
Evals (8.4) catch problems before you ship; runtime guardrails catch them as they happen. Think of a guardrail sandwich: validate the input going in, constrain the model, then validate the output coming out — before it reaches a user or triggers a side effect. Neither side alone is enough: input checks stop obvious abuse, output checks catch what the model generated anyway.
| Stage | Guardrail | Example |
|---|---|---|
| Input | Validation + moderation pass | Reject oversized/malformed input; classify prompts for policy before they reach the model |
| Model | Ground & constrain | Allow "I don't know"; restrict to provided context; require citations |
| Output | Validation + classification pass | Schema/format check; a moderation model screens the response before display |
| Actions | Allow-list tools · spend/rate caps · human-in-the-loop | Only whitelisted tools; per-tenant token cap; confirm irreversible actions |
CONCEPT Bound the blast radius of actions
An agent that can call tools can cause real-world harm, so bound what it's allowed to do.
Allow-list tools — deny-by-default, expose only the tools this agent needs. Impose
spend and rate caps so a runaway loop can't burn the budget or blow past your OTPM limit (8.3).
And keep a human in the loop for risky, irreversible actions — the model proposes, a person
approves before send_email, delete, or a payment executes. These are the same
reversibility-and-gating instincts from D1, applied as production controls.
DIAGRAM The guardrail sandwich
+ passes moderation?"} IV -->|"no"| REJ["🔴 Reject early"] IV -->|"yes"| M["Model call
grounded · may say 'I don't know'"] M --> OV{"Output valid
+ passes classifier?"} OV -->|"no"| BLOCK["🔴 Block / retract claim"] OV -->|"yes"| ACT{"Action requested?"} ACT -->|"tool not allow-listed
or over spend cap"| DENY["🔴 Deny"] ACT -->|"risky / irreversible"| HITL["🟠 Human-in-the-loop
confirm"] ACT -->|"safe + within caps"| RUN["🟢 Execute"] HITL --> RUN classDef ok fill:#d4edda,stroke:#28a745,color:#155724; classDef gate fill:#ffe8cc,stroke:#ff9f40,color:#9a4a0a; classDef bad fill:#f8d7da,stroke:#dc3545,color:#721c24; class RUN ok; class HITL gate; class REJ bad; class BLOCK bad; class DENY bad;
CODE Input check → model → output check → gated action
Deny-by-default at every seam: moderate input, validate output, allow-list the tool, enforce a spend cap, and gate irreversible actions behind a human.
ALLOWED_TOOLS = {"search", "read_file"} # deny-by-default allow-list
IRREVERSIBLE = {"send_email", "delete_file", "charge_card"}
def handle(user_input):
if not moderate(user_input): # input moderation pass
return "Refused: input violates policy."
if spend.today > SPEND_CAP: # spend/rate cap
return "Halted: daily budget reached."
resp = client.messages.create(model="claude-opus-4-8", max_tokens=1024,
system=GROUNDED_SYSTEM, # "use only the docs; you may say I don't know"
messages=[{"role": "user", "content": user_input}])
if not validate_output(resp) or not classify_safe(resp):
return "Blocked: output failed validation." # output guardrail
for tool in tool_calls(resp):
if tool.name not in ALLOWED_TOOLS:
return f"Denied: '{tool.name}' not allow-listed."
if tool.name in IRREVERSIBLE and not human_approves(tool):
return f"Denied: '{tool.name}' needs human approval." # HITL
run_tool(tool)
TRADE-OFFS Guardrail coverage vs. latency & friction
✅ Layered guardrails
- Input + output classification catches abuse both ways
- Allow-lists + caps bound the blast radius
- HITL stops irreversible mistakes cold
⚠️ The cost of layering
- Extra moderation calls add latency & token cost
- HITL on everything defeats autonomy — gate by risk
- Over-tight input filters reject legitimate users
EXAM TRAPS
PRACTICE
Q1 · Why validate output when you already validate input?
Input checks stop bad requests, but the model can still generate unsafe, hallucinated, or malformed output from a perfectly benign prompt. A separate output validation / classification pass screens the response before it reaches a user or triggers an action — you need both sides of the sandwich.
Q2 · Three runtime controls that bound what an agent's tool calls can do?
Allow-list the tools (deny-by-default, expose only what's needed); spend/rate caps so a runaway loop can't drain budget or blow the OTPM limit; and human-in-the-loop approval for risky, irreversible actions before they execute.
Q3 · Which prompt-level techniques reduce hallucination at runtime?
Explicitly allow the model to say "I don't know," restrict it to the provided documents rather than general knowledge, and require it to cite quotes so each claim is auditable and unsupported ones are retracted. Back these with an output-verification layer for high-stakes use.