Security & Policy
Data Poisoning
Corrupting a model training data on purpose to plant hidden flaws or backdoors in the resulting model.
Data poisoning attacks the model before it is even built, by slipping manipulated examples into the training data so the trained model misbehaves in ways the attacker chooses, sometimes only when a secret trigger appears. Because modern models train on enormous, loosely vetted web data, even a small amount of poisoned content can have an effect. Defenses center on vetting data sources and detecting anomalous training examples.
Related terms
Training Data
The examples a model learns from, whose size, quality, and bias largely determine how the model turns out.
Algorithmic Bias
When an AI system produces unfair or skewed results, usually because its training data reflected existing bias.
Red Teaming
Deliberately probing an AI system to find ways it can fail, be misused, or produce harmful output, before attackers do.
Guardrails
Controls placed around a model to keep its inputs and outputs safe, on-topic, and within policy.