Prompting & Inference
Context Window
The maximum amount of text, measured in tokens, that a model can consider at once.
The context window is the models working memory for a single request: everything it can take into account, including your prompt, the conversation so far, and any documents you paste in. Once a conversation exceeds that limit, the earliest material has to be dropped or summarized. Larger context windows let a model handle longer documents and conversations, which is why window size has become a headline spec for models.
Reference: Wikipedia: Context Window
Related terms
Token
The small chunk of text a language model actually reads and writes, usually a word or part of a word.
Attention
A mechanism that lets a model focus on the most relevant parts of its input when producing each piece of output.
Retrieval-Augmented Generation
A technique that looks up relevant documents and feeds them to a model so its answers are grounded in real sources.
Large Language Model
A model trained on massive amounts of text that predicts and generates language, powering tools like ChatGPT and Claude.