Alignment & Safety
Constitutional AI
A training approach that aligns a model to an explicit written set of principles instead of relying only on human ratings.
Constitutional AI guides a models behavior using a stated set of principles, a constitution, that the model uses to critique and revise its own responses during training. It reduces the need for humans to label huge numbers of harmful examples by hand, and it makes the values being trained in more explicit and reviewable. It is one practical method for alignment, developed by Anthropic.
Related terms
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.
Reinforcement Learning from Human Feedback
A training method that tunes a model to be helpful by rewarding the responses human reviewers prefer.
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.