Security & Policy
Model Collapse
The degradation that can occur when models are trained on AI-generated data, generation after generation, losing touch with reality.
Model collapse describes how quality can erode when AI output is fed back in as training data for the next model, repeatedly. Each generation overfits to the last one quirks and loses the rare cases and variety of genuine human data, drifting toward blander or distorted output. As AI content floods the web the concern is real, though recent work argues it is avoidable as long as fresh human data keeps accumulating alongside the synthetic.
Reference: Wikipedia: Model Collapse
Related terms
Training Data
The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.
Generative AI
AI that creates new content such as text, images, audio, or code, rather than just classifying or scoring existing data.
Overfitting
When a model memorizes its training data too closely and then performs badly on anything new.
Data Poisoning
Corrupting a model training data on purpose to plant hidden flaws or backdoors in the resulting model.