S4
Guardrails and Tripwires: Real Vetoes, Not Polite Prompts
Put independent veto layers around the model instead of relying on instructions it can ignore.
- 01Distinguish a guardrail (separate validation layer) from a system-prompt instruction
- 02Place input guardrails (injection/PII/policy) before the model and output guardrails (schema/moderation/grounding) before downstream consumers
- 03Explain how a tripwire aborts a run and why guardrails run as cheap parallel classifiers
- 04Reason about the strictness tradeoff: abuse blocked versus false positives and added latency
A guardrail is a layer, not a sentence
topic 11Guardrails are separate validation layers that wrap the model, distinct from anything written in the prompt:
- Input guardrails run before the model: prompt-injection detection, PII screening, policy checks.
- Output guardrails run before downstream consumers: schema validation, moderation, grounding/faithfulness checks.
When a check fails hard, a tripwire aborts the run so tainted output never propagates. (“Tripwire” is the OpenAI Agents SDK’s term for exactly this abort mechanism.) Implement them as cheap classifiers that run in parallel so they add minimal latency.
Why the prompt is not a boundary
A system-prompt instruction shares the same token stream as user and tool content; an injection or jailbreak can override it. A guardrail living in separate code cannot be argued with — it either passes the payload or trips. This is the same principle behind input/output separation: trust is enforced by structure, not by wording.
Common MisconceptionGuardrails are just careful prompt wording.
CorrectionA guardrail is a separate veto layer (classifier/validator) outside the model; prompt wording is a suggestion the model can be overridden out of.
The strictness tradeoff
| Tighter guardrails | Looser guardrails |
|---|---|
| More abuse and bad output blocked | Fewer false positives, less friction |
| More false positives + added latency/cost | More injection/PII/policy leakage slips through |
Tune the threshold to the blast radius of a miss: an outbound-email tool warrants stricter screening than a read-only summarizer.
Production failure modes
- Guardrail false negatives: an injection or PII leak passes an under-tuned classifier.
- Guardrail false positives: legitimate requests blocked, frustrating users and adding cost.
- Output guardrail missing: malformed or ungrounded output reaches a downstream tool or DB.
- Guardrails run serially in the critical path, adding avoidable latency per hop.
Product names in this space (Llama Guard, NeMo Guardrails, Guardrails AI) are illustrative — the pattern matters more than the vendor.
References & deeper reading (1)
- GuardrailsOpenAI Agents SDK
Retrieval Practice
Check one idea at a time
Your system prompt says "never output a user's email address." A crafted request still extracts one. What is the actual fix?