Security & Policy

Prompt Injection

An attack that hides malicious instructions in content a model reads, tricking it into ignoring its real orders.

Prompt injection exploits the fact that a model cannot always tell its trusted instructions apart from the text it is processing. An attacker plants commands in a web page, email, or document the model ingests, and the model may follow them, leaking data or taking unwanted actions. It is widely considered the top security risk for LLM applications, and it is especially dangerous for agents that can act on the world, with no fully reliable fix yet.

Reference: Wikipedia: Prompt Injection