Prompting & Inference

Temperature

A setting that controls how random or predictable a model’s output is.

Temperature tunes how much a model gambles when choosing each next token. Low temperature makes it pick the most likely option almost every time, giving focused, repeatable, sometimes dull output, while high temperature loosens things up for more variety and creativity at the risk of wandering. It is one of the simplest dials for matching a models behavior to the task, factual versus imaginative.