Models & Architecture
Recurrent Neural Network
RNNAn older network design for sequences like text or audio, which processes one step at a time while carrying a memory forward.
An RNN reads a sequence element by element, keeping a running internal state that acts as memory of what it has seen so far. That made it a natural fit for language and time-series data before transformers arrived. RNNs struggle to hold onto information across long sequences, which is one of the main problems the transformer and its attention mechanism were designed to solve.
Reference: Wikipedia: Recurrent Neural Network
Related terms
Transformer
The neural network design behind almost every modern language model, built around a mechanism called attention.
Neural Network
A model loosely inspired by the brain, built from layers of simple units that each pass signals to the next.
Convolutional Neural Network
A network design built for images, which scans for small visual patterns and builds them up into recognizable objects.