Models & Architecture

Recurrent Neural Network

RNN

An older network design for sequences like text or audio, which processes one step at a time while carrying a memory forward.

An RNN reads a sequence element by element, keeping a running internal state that acts as memory of what it has seen so far. That made it a natural fit for language and time-series data before transformers arrived. RNNs struggle to hold onto information across long sequences, which is one of the main problems the transformer and its attention mechanism were designed to solve.

Reference: Wikipedia: Recurrent Neural Network