Models & Architecture
Activation Function
The small nonlinear function applied at each neuron that lets a network learn complex, nonlinear patterns.
An activation function decides how strongly each neuron fires based on its input, and crucially it is nonlinear, which is what lets a deep network model curves and interactions rather than just straight lines. Common choices include ReLU, which passes positive values and zeros out the rest, and sigmoid and softmax for producing probabilities. Without them, stacking layers would collapse into a single linear step.
Reference: Wikipedia: Activation Function
Related terms
Neural Network
A model loosely inspired by the brain, built from layers of simple units that each pass signals to the next.
Deep Learning
Machine learning built on large neural networks with many layers, which is what powers most recent AI breakthroughs.
Weights
The individual strengths of the connections in a neural network, which are what actually change as a model learns.