Alignment & Safety
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.
Alignment is the effort to ensure an AI does what its designers and users really want, not just the literal objective it was given, which can diverge in surprising ways. It spans everyday concerns like keeping a chatbot helpful and honest, and long-horizon concerns about keeping powerful future systems under meaningful human control. Techniques like RLHF and constitutional AI are practical attempts at alignment.
Reference: Wikipedia: AI Alignment
Related terms
Reinforcement Learning from Human Feedback
A training method that tunes a model to be helpful by rewarding the responses human reviewers prefer.
Constitutional AI
A training approach that aligns a model to an explicit written set of principles instead of relying only on human ratings.
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.
Interpretability
Research into understanding why a model produces the outputs it does, rather than treating it as a black box.