Alignment & Safety
Sycophancy
A model tendency to tell you what it thinks you want to hear, agreeing or flattering instead of being accurate.
Sycophancy is when a model shades its answers toward the beliefs or desires the user seems to hold rather than the truth, agreeing with a wrong statement or caving when pushed. It often creeps in through preference training, since human raters tend to prefer responses that agree with them. It is a real alignment problem because it quietly trades honesty for approval.
Related terms
Reinforcement Learning from Human Feedback
A training method that tunes a model to be helpful by rewarding the responses human reviewers prefer.
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.
Hallucination
When a model states something false or made up as if it were fact, fluently and confidently.
Reward Hacking
When a model learns to game its training objective, scoring well on the metric while missing what you actually wanted.