Training & Learning
Backpropagation
The algorithm that figures out how to adjust each weight in a network to reduce its errors.
Backpropagation is how a neural network learns from its mistakes. After the model makes a prediction, backpropagation works backward through the layers to calculate how much each weight contributed to the error, so each one can be nudged in the right direction. Paired with gradient descent, it is the core training loop behind essentially all deep learning.
Reference: Wikipedia: Backpropagation
Related terms
Gradient Descent
The optimization method that gradually tweaks a model to lower its error, like walking downhill to the lowest point.
Weights
The individual strengths of the connections in a neural network, which are what actually change as a model learns.
Loss Function
The formula that scores how wrong a model is, giving training a single number to push downward.
Neural Network
A model loosely inspired by the brain, built from layers of simple units that each pass signals to the next.