Alignment & Safety
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.
Guardrails are the checks and limits wrapped around an AI system to prevent unwanted behavior, such as filters that block harmful requests, rules that keep a bot on topic, or validators that catch unsafe outputs before they reach a user. They sit outside the models own training as an extra layer of defense. Good guardrails are essential for deploying AI in production, especially anywhere the output touches customers or money.
Related terms
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.
Hallucination
When a model states something false or made up as if it were fact, fluently and confidently.
Jailbreak
A prompt crafted to trick a model into bypassing its own safety rules and doing something it should refuse.
Red Teaming
Deliberately probing an AI system to find ways it can fail, be misused, or produce harmful output, before attackers do.