Training & Learning

Learning Rate

The size of the steps a model takes while learning, and one of the most important training settings to get right.

The learning rate controls how big an adjustment the model makes to its weights on each training step. Too high and training overshoots and becomes unstable, too low and it crawls or gets stuck. Finding a good value, and often decreasing it over the course of training, is one of the most consequential parts of tuning a model.

Reference: Wikipedia: Learning Rate