Training & Learning
Training Data
The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.
Training data is the collection of examples used to teach a model, and it shapes almost everything about the result. A model can only learn what is represented in its data, so gaps and biases in the data become gaps and biases in the model. For large language models the data is a huge slice of text, and curating it well is one of the highest-leverage parts of building a good model.
Reference: Wikipedia: Training Data
Related terms
Pretraining
The first, largest training phase where a model learns general patterns from a huge amount of broad data.
Supervised Learning
Training a model on labeled examples, where each input comes paired with the correct answer.
Algorithmic Bias
When an AI system produces unfair or skewed results, usually because its training data reflected existing bias.
Overfitting
When a model memorizes its training data too closely and then performs badly on anything new.