Training & Learning

Gradient Descent

The optimization method that gradually tweaks a model to lower its error, like walking downhill to the lowest point.

Gradient descent improves a model by repeatedly taking small steps in the direction that reduces its error the fastest. The gradient points uphill toward more error, so the method steps the opposite way, downhill, over and over. The size of each step is called the learning rate, and choosing it well is one of the everyday arts of training models.

Reference: Wikipedia: Gradient Descent