Prompting & Inference
Prompt
The input you give a generative model, usually instructions or a question, that tells it what you want.
A prompt is whatever you feed a model to get a response, from a one-line question to a long set of instructions with examples and context. Because the same model can do wildly different things depending on how you ask, the prompt is the main steering wheel for everyday users. Writing prompts well enough to get reliable results is a skill in its own right, called prompt engineering.
Related terms
Prompt Engineering
The practice of wording and structuring prompts to get better, more reliable results from a model.
System Prompt
A behind-the-scenes instruction that sets a model’s role, rules, and tone for an entire conversation.
Context Window
The maximum amount of text, measured in tokens, that a model can consider at once.
Chain-of-Thought
Prompting a model to reason step by step, which tends to improve its accuracy on hard problems.