Governance
Guardrails
Definition
Guardrails are the deterministic checks around a probabilistic system — input filters, output validators, scope limits, and refusal paths that do not depend on the model behaving well.
A guardrail that is itself a prompt is a suggestion. Real guardrails are code: schema validation, allow-lists, monetary thresholds, permission checks, and hard stops that execute regardless of what the model decided to output. They are what lets a system that is right ninety-five percent of the time be safe one hundred percent of the time, which is the gap between a demo and a deployable production system.
Deterministic beats persuasive
Validating that a returned policy number exists in your system is a control that either passes or fails with no ambiguity. Asking the model to only use real policy numbers is a hope, dressed up in careful prompt language, that the model will behave the way you asked every single time, including on the inputs you never tested against. The difference between these two approaches is the difference between a system you can certify and one you can only vouch for.
Design the refusal
The most valuable guardrail behaviour is a clean stop with a route to a human, delivered before an uncertain answer reaches a customer or a decision. Systems without a defined refusal path invent an answer instead, because the underlying model is built to produce plausible continuations and has no natural mechanism to simply say nothing when it is genuinely unsure, which means the refusal behavior has to be engineered in deliberately rather than assumed to happen on its own.
Guardrails need their own testing
A guardrail that has never been deliberately triggered in testing is a guardrail you are trusting rather than one you have verified works. Building specific test cases designed to hit each threshold, each validator, and each refusal path before launch confirms the control actually fires as intended, rather than discovering during a live incident that a hard stop had an edge case nobody anticipated, which is a far more expensive way to find the gap than a dedicated pre-launch test suite.
In practice
- Validate outputs against a schema and against your systems of record
- Set hard thresholds for value, volume, and irreversibility
- Make 'I cannot answer this' a first-class, well-handled outcome
- Build dedicated test cases that deliberately trigger every guardrail before launch
