Models & Architecture
Long Short-Term Memory
LSTMA type of recurrent network with a built-in memory gate, designed to remember information across long sequences.
LSTM is a recurrent neural network built to solve the problem of forgetting: ordinary RNNs lose track of information over long sequences, and the LSTM adds a gated memory cell that decides what to keep, update, or discard at each step. For years it was the standard for language, speech, and time-series tasks. Transformers have largely replaced it for large-scale language work, but it is still used where sequences are long and data is limited.
Reference: Wikipedia: Long Short-Term Memory
Related terms
Recurrent Neural Network
An older network design for sequences like text or audio, which processes one step at a time while carrying a memory forward.
Transformer
The neural network design behind almost every modern language model, built around a mechanism called attention.
Attention
A mechanism that lets a model focus on the most relevant parts of its input when producing each piece of output.