Alignment & Safety
Interpretability
Research into understanding why a model produces the outputs it does, rather than treating it as a black box.
Interpretability is the effort to see inside a model and explain its behavior, from which inputs drove a decision to what concepts its internal units represent. It matters for trust, debugging, and safety, since it is hard to rely on a system nobody can explain. Large neural networks are notoriously opaque, so this is an active and difficult research area.
Reference: Wikipedia: Interpretability
Related terms
AI Alignment
The work of making AI systems pursue what people actually intend, including their values and safety.
AI Ethics
The study and practice of building and using AI responsibly, covering fairness, privacy, accountability, and harm.
Neural Network
A model loosely inspired by the brain, built from layers of simple units that each pass signals to the next.