🎯 A single catalog of every guardrail worth building, grouped by risk. Each
GR line is written as a thing to develop with the
layer it must be enforced at. Cost links to its own page; the rest are here.
The one principle behind all of them: a guardrail that must hold belongs in
deterministic code / permissions / infra — never in the prompt. A prompt can be jailbroken or drift; a read-only
grant, an approval gate, or a schema check cannot. Layer them (defense-in-depth): the model is one layer, not the fence.
Layer legend
INFRA network/identity/sandbox · TOOL tool & permission layer ·
APP your orchestration code · MODEL model/API feature ·
PROMPT instructions (weakest — never rely on alone).
💰 Cost🔐 Security & Access💉 Prompt Injection
🌫️ Hallucination🛡️ Reliability🕵️ Privacy / GxP
🚫 Content Safety🧑⚖️ Human-in-the-Loop📊 Observability
💰 COST & BUDGET → dedicated page
The full cost-guardrail set (R1–R10: uncached prefix, cache-miss detection, oversized model, output ceiling, effort
mismatch, history bloat, prose-vs-schema, unbatched bulk, unbounded loop, deprecated params) lives on the
Cost — Anti-Patterns, Best Practices & Guardrails page. In short: enforce a
per-request & per-session token budget APP, a hard spend cap / kill-switch
INFRA, and alert on cache-hit-rate collapse APP.
🔐 SECURITY & ACCESS least privilege, always
Risk: an agent (or a tricked agent) does something it was never meant to — read, write, or reach a system it
shouldn't. Safety here is structural, not persuasive.
GR-SEC-1 · Read-only by default — every data tool uses a read-only grant / role; writes are a separate, explicitly-granted tool. TOOL
GR-SEC-2 · Least-privilege service principal — scope credentials to exactly the tables/resources needed; no broad admin tokens. INFRA
GR-SEC-3 · Tool allow-list — the agent can only call an explicit set of tools; deny-by-default. TOOL
GR-SEC-4 · Query/action constraints — statement timeouts, row/byte caps, no DDL/DML, allow-listed schemas. TOOL
GR-SEC-5 · Secret hygiene — secrets from a vault/scope, never in prompts, code, or logs; redact before the model. INFRA
GR-SEC-6 · Egress control — restrict outbound network from tool sandboxes; no arbitrary URLs. INFRA
GR-SEC-7 · Sandbox untrusted execution — code/bash tools run isolated, ephemeral, resource-capped. INFRA
GR-SEC-8 · Confused-deputy prevention — carry the user's permissions into tool calls; don't let a low-priv user act through a high-priv agent identity. TOOL
Anti-pattern: "only read data, never delete" written in the system prompt. That's not a control — a read-only grant is. Security lives in the permission layer.
💉 PROMPT INJECTION & UNTRUSTED CONTENT
Risk: tool results, retrieved docs, emails, logs, or web pages carry instructions ("ignore your rules and…") and
the model follows them. The request was valid; the data hijacked it.
GR-INJ-1 · Treat all tool/retrieved content as DATA, never instructions — isolate it (e.g. XML-delimited) from the instruction channel. PROMPT APP
GR-INJ-2 · Least-privilege tools — even a fully-hijacked agent can't do damage it has no tool/permission for. TOOL
GR-INJ-3 · Sanitize / spotlight inputs — mark provenance ("the following is untrusted content"); strip/flag embedded instructions. APP
GR-INJ-4 · Human approval on sensitive actions triggered from untrusted data — a tool call that originated from retrieved content needs a gate. APP
GR-INJ-5 · Output filtering — scan responses for exfiltration attempts (leaked secrets, unexpected URLs/emails) before acting on them. APP
GR-INJ-6 · Don't echo untrusted content into a privileged context — avoid feeding raw tool output straight into a step with write powers. APP
🌫️ HALLUCINATION & GROUNDING
Risk: a fluent, confident, wrong answer returned as a normal 200/end_turn — the
silent-failure class nothing in the API flags.
GR-HAL-1 · Ground on provided context — "answer only from the supplied sources; if absent, say not found." PROMPT
GR-HAL-2 · Require citations — every claim cites a source/evidence id; reject uncited claims. APP MODEL
GR-HAL-3 · Verifier pass (fresh context) — an independent check that every claim is supported before the output is trusted. APP
GR-HAL-4 · Structured output + VALUE validation — schema guarantees shape, not truth; validate ranges, enums, referential integrity, and that cited ids are real. MODEL APP
GR-HAL-5 · Deterministic checks where they exist — EXPLAIN/dry-run a query, recompute a number, cross-check against a source of truth. APP
GR-HAL-6 · Allow "I don't know" — an explicit low-confidence / abstain path beats a confident guess. PROMPT
GR-HAL-7 · Confidence + self-consistency — report confidence; sample N and take the majority on high-stakes answers. APP
GR-HAL-8 · Golden-set evals — assert field values / faithfulness on a fixed set, run on every prompt/model change. APP
🛡️ RELIABILITY & RESILIENCE
Risk: transient errors, timeouts, truncation, or runaway loops turn into outages, double-billing, or corrupted state.
GR-REL-1 · Retry only the retryable — 429 & 5xx with exponential backoff + jitter (honor Retry-After); never retry a 400. APP
GR-REL-2 · Idempotency — side-effecting tools use idempotency keys / dedup so a retry can't double-execute. TOOL
GR-REL-3 · Timeouts on every tool & call — nothing hangs indefinitely. APP
GR-REL-4 · Handle every stop_reason — route tool_use; treat max_tokens as truncated (continue, don't accept); handle refusal; catch-all for the rest. APP
GR-REL-5 · Bound the loop — max_turns + graceful "couldn't complete" (no runaway cost/actions). APP
GR-REL-6 · Circuit breakers & fallbacks — trip on repeated failures; fall back to a smaller model / cached / degraded answer. APP
GR-REL-7 · Graceful degradation — partial results beat a hard failure; always return something actionable. APP
GR-REL-8 · Stream long outputs — avoid request timeouts (and the retry double-bill). APP
GR-REL-9 · Fail-open on non-critical AI — if narration/triage errors, still emit the deterministic result; never let the AI layer swallow a real signal. APP
🕵️ PRIVACY, PII & COMPLIANCE (GxP)
Risk: sensitive data reaches the model, the logs, or an output where it shouldn't — a regulatory and trust failure.
GR-PRV-1 · PII redaction before the model — detect & mask PII/PHI in inputs, tool results, and logs. APP
GR-PRV-2 · Data minimization — send only the fields the task needs; don't dump whole rows/records. APP
GR-PRV-3 · Retention / ZDR — configure zero-data-retention or short retention where required; know the region. INFRA
GR-PRV-4 · Immutable audit trail — append-only log of inputs (hashed), outputs, model+prompt version, and the approver — reproducible & provable. APP
GR-PRV-5 · Access-scope the corpus — RAG/memory only surfaces what the requesting user is allowed to see. TOOL
GR-PRV-6 · No PII in the agent's own logs/traces — scrub before persisting telemetry. APP
GR-PRV-7 · Nothing makes a regulated decision autonomously — GxP-affecting outcomes require a human of record. APP
🚫 CONTENT SAFETY & OUTPUT
Risk: harmful, off-policy, or malformed output reaches users or downstream systems.
GR-SAF-1 · Handle refusal gracefully — a safety-declined response is a 200; branch on it, show a clean message. MODEL APP
GR-SAF-2 · Input/output moderation — screen user input and model output against your content policy (add a classifier if needed). APP
GR-SAF-3 · Output schema validation — validate/parse before anything consumes the output; reject malformed. APP
GR-SAF-4 · Topic / scope boundaries — keep the agent within its remit; refuse out-of-scope asks. PROMPT APP
GR-SAF-5 · No sensitive data in outputs — filter secrets/PII from what's returned to the user. APP
🧑⚖️ HUMAN-IN-THE-LOOP & AUTONOMY
Risk: the agent takes an irreversible or high-impact action it shouldn't have taken alone.
GR-HITL-1 · Approval gate on high-stakes / irreversible actions — writes, deletes, spend, access changes require explicit human sign-off. APP
GR-HITL-2 · Autonomy ladder — start Observe→Explain→Recommend; graduate to Act only for mature, low-risk actions. APP
GR-HITL-3 · Action allow-list + dry-run — actionable tools are enumerated; preview the effect before executing. TOOL
GR-HITL-4 · Rollback / undo — every write path has a reversal; prefer reversible actions. APP
GR-HITL-5 · Approver of record + reason — log who approved what and why (feeds the audit trail). APP
📊 OBSERVABILITY & GOVERNANCE the meta-guardrail
Risk: you can't guardrail what you can't see. Observability is what makes every other guardrail measurable and every
silent failure catchable.
GR-OBS-1 · Trace every step — per-request run-id, each tool call, tokens, latency, stop_reason, cost. APP
GR-OBS-2 · Evals in production — sampled quality/faithfulness checks; alert on regression & drift. APP
GR-OBS-3 · SLOs & alerting — success rate, p95 latency, error/refusal rate, cost/req; page on breach. APP
GR-OBS-4 · Rate limiting & quotas — per-user/tenant caps to contain abuse and cost. INFRA
GR-OBS-5 · Kill switch — one control to disable an agent/tool instantly. INFRA
GR-OBS-6 · Version pinning — pin model + prompt + tool versions; log them so any output is reproducible. APP
COVERAGE MATRIX category → where enforced → CCA domain
| Guardrail category | Primary layer | Maps to CCA |
| 💰 Cost & budget | App + Infra | D5 |
| 🔐 Security & access | Tool + Infra | D1 · D2 |
| 💉 Prompt injection | App + Tool (never prompt alone) | D2 · D3 |
| 🌫️ Hallucination / grounding | App + Model | D4 · D5 |
| 🛡️ Reliability | App | D5 · D1 |
| 🕵️ Privacy / GxP | App + Infra | D5 · D2 |
| 🚫 Content safety | Model + App | D5 · D4 |
| 🧑⚖️ Human-in-the-loop | App | D1 |
| 📊 Observability | App + Infra | D5 |
Build order suggestion: Security & Reliability first (they prevent the worst outcomes),
then Grounding + HITL (correctness & safe action), then Privacy/GxP (for regulated work), with Observability wired in
from day one because it's what proves the rest are working.