Models & Architecture
Transformer
The neural network design behind almost every modern language model, built around a mechanism called attention.
The transformer is the architecture introduced in 2017 that made todays large language models possible. Its key trick is attention, which lets the model weigh how much every word in a sequence relates to every other word, in parallel, instead of reading strictly left to right. That parallelism made it practical to train on far more data than earlier designs, and it now underpins text, image, and audio models alike.
Reference: Wikipedia: Transformer
Related terms
Attention
A mechanism that lets a model focus on the most relevant parts of its input when producing each piece of output.
Large Language Model
A model trained on massive amounts of text that predicts and generates language, powering tools like ChatGPT and Claude.
Deep Learning
Machine learning built on large neural networks with many layers, which is what powers most recent AI breakthroughs.
Parameters
The adjustable numbers inside a model that get tuned during training and store what the model has learned.