Alignment & Safety
Jailbreak
A prompt crafted to trick a model into bypassing its own safety rules and doing something it should refuse.
A jailbreak is an input designed to get around a models safety training, coaxing it to produce content it is supposed to decline, often through role-play, misdirection, or clever framing. It is the AI equivalent of social engineering, aimed at the model rather than a person. Labs study jailbreaks through red teaming and keep hardening models against them, in an ongoing back-and-forth.
Related terms
Red Teaming
Deliberately probing an AI system to find ways it can fail, be misused, or produce harmful output, before attackers do.
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.
Prompt
The input you give a generative model, usually instructions or a question, that tells it what you want.