Security & Policy

Data Poisoning

Corrupting a model training data on purpose to plant hidden flaws or backdoors in the resulting model.

Data poisoning attacks the model before it is even built, by slipping manipulated examples into the training data so the trained model misbehaves in ways the attacker chooses, sometimes only when a secret trigger appears. Because modern models train on enormous, loosely vetted web data, even a small amount of poisoned content can have an effect. Defenses center on vetting data sources and detecting anomalous training examples.