Training & Learning
Optimizer
The algorithm that decides how to adjust a model weights from its errors, with Adam being the common default.
An optimizer takes the error signal from backpropagation and turns it into actual weight updates, deciding the direction and size of each step. Plain gradient descent is the baseline, while adaptive methods like Adam adjust the step for each weight automatically and converge faster, which is why Adam is the default for most deep learning. The choice of optimizer and its settings meaningfully affects how well and how quickly a model trains.
Related terms
Gradient Descent
The optimization method that gradually tweaks a model to lower its error, like walking downhill to the lowest point.
Backpropagation
The algorithm that figures out how to adjust each weight in a network to reduce its errors.
Learning Rate
The size of the steps a model takes while learning, and one of the most important training settings to get right.
Loss Function
The formula that scores how wrong a model is, giving training a single number to push downward.