Training & Learning
Loss Function
The formula that scores how wrong a model is, giving training a single number to push downward.
A loss function measures the gap between what a model predicted and what it should have predicted, boiling the error down to one number. Training is essentially the search for the weights that make this number as small as possible. The choice of loss function defines what the model is actually optimizing for, which is why picking the wrong one can quietly steer a model toward the wrong behavior.
Reference: Wikipedia: Loss Function
Related terms
Gradient Descent
The optimization method that gradually tweaks a model to lower its error, like walking downhill to the lowest point.
Backpropagation
The algorithm that figures out how to adjust each weight in a network to reduce its errors.
Overfitting
When a model memorizes its training data too closely and then performs badly on anything new.