Models & Architecture
Dropout
A training trick that randomly switches off some neurons each step to stop a network from overfitting.
Dropout fights overfitting by randomly ignoring a fraction of neurons on each training step, so the network cannot lean too hard on any single unit and has to spread what it learns. The effect is a bit like training many slightly different networks and averaging them. At inference time all neurons are active, giving a more robust model.
Reference: Wikipedia: Dropout
Related terms
Overfitting
When a model memorizes its training data too closely and then performs badly on anything new.
Regularization
Any technique that discourages a model from fitting its training data too closely, to help it generalize.
Neural Network
A model loosely inspired by the brain, built from layers of simple units that each pass signals to the next.
Training Data
The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.