Security & Policy

Model Collapse

The degradation that can occur when models are trained on AI-generated data, generation after generation, losing touch with reality.

Model collapse describes how quality can erode when AI output is fed back in as training data for the next model, repeatedly. Each generation overfits to the last one quirks and loses the rare cases and variety of genuine human data, drifting toward blander or distorted output. As AI content floods the web the concern is real, though recent work argues it is avoidable as long as fresh human data keeps accumulating alongside the synthetic.

Reference: Wikipedia: Model Collapse