Alignment & Safety
Red Teaming
Deliberately probing an AI system to find ways it can fail, be misused, or produce harmful output, before attackers do.
Red teaming is adversarial testing: people try hard to break a model, trick it into unsafe answers, or expose weaknesses, so those problems can be fixed before release. It is borrowed from security practice and has become standard for frontier models. The findings feed back into better guardrails and training, and the term red team refers to the people playing attacker.
Reference: Wikipedia: Red Teaming
Related terms
Jailbreak
A prompt crafted to trick a model into bypassing its own safety rules and doing something it should refuse.
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.