Training & Learning

Training Data

The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.

Training data is the collection of examples used to teach a model, and it shapes almost everything about the result. A model can only learn what is represented in its data, so gaps and biases in the data become gaps and biases in the model. For large language models the data is a huge slice of text, and curating it well is one of the highest-leverage parts of building a good model.

Reference: Wikipedia: Training Data