Foundations
Self-Supervised Learning
SSLA training approach where the data provides its own labels, like hiding part of an input and predicting it, which is how LLMs first learn language.
Self-supervised learning sits between supervised and unsupervised: there are no human labels, but the model still gets a concrete target generated automatically from the data itself. For text that usually means hiding a word and predicting it from the surrounding context, repeated across billions of examples. It is the workhorse behind large language models, since it lets them learn from raw internet text without anyone labeling it by hand.
Reference: Wikipedia: Self-Supervised Learning
Related terms
Unsupervised Learning
Training a model on data with no labels, so it has to find structure and patterns on its own.
Supervised Learning
Training a model on labeled examples, where each input comes paired with the correct answer.
Pretraining
The first, largest training phase where a model learns general patterns from a huge amount of broad data.
Large Language Model
A model trained on massive amounts of text that predicts and generates language, powering tools like ChatGPT and Claude.