Prompting & Inference

Context Window

The maximum amount of text, measured in tokens, that a model can consider at once.

The context window is the models working memory for a single request: everything it can take into account, including your prompt, the conversation so far, and any documents you paste in. Once a conversation exceeds that limit, the earliest material has to be dropped or summarized. Larger context windows let a model handle longer documents and conversations, which is why window size has become a headline spec for models.

Reference: Wikipedia: Context Window