Training & Learning
Regularization
Any technique that discourages a model from fitting its training data too closely, to help it generalize.
Regularization is the family of methods that keep a model from memorizing noise and overfitting, usually by penalizing complexity so it prefers simpler explanations. Classic approaches add a penalty on large weights, while dropout and early stopping are others. The goal is always the same: better performance on new data, even at a small cost on the training data.
Reference: Wikipedia: Regularization
Related terms
Overfitting
When a model memorizes its training data too closely and then performs badly on anything new.
Dropout
A training trick that randomly switches off some neurons each step to stop a network from overfitting.
Loss Function
The formula that scores how wrong a model is, giving training a single number to push downward.
Hyperparameter
A setting you choose before training that controls how the model learns, as opposed to what it learns.