Prompting & Inference
Temperature
A setting that controls how random or predictable a model’s output is.
Temperature tunes how much a model gambles when choosing each next token. Low temperature makes it pick the most likely option almost every time, giving focused, repeatable, sometimes dull output, while high temperature loosens things up for more variety and creativity at the risk of wandering. It is one of the simplest dials for matching a models behavior to the task, factual versus imaginative.
Related terms
Inference
Actually running a trained model to get an answer, as opposed to training it in the first place.
Prompt
The input you give a generative model, usually instructions or a question, that tells it what you want.
Hallucination
When a model states something false or made up as if it were fact, fluently and confidently.