Alignment & Safety

Interpretability

Research into understanding why a model produces the outputs it does, rather than treating it as a black box.

Interpretability is the effort to see inside a model and explain its behavior, from which inputs drove a decision to what concepts its internal units represent. It matters for trust, debugging, and safety, since it is hard to rely on a system nobody can explain. Large neural networks are notoriously opaque, so this is an active and difficult research area.

Reference: Wikipedia: Interpretability