Prompting & Inference

Top-p Sampling

A decoding method that picks the next token from the smallest set of options whose probabilities add up to a chosen threshold.

Top-p sampling, also called nucleus sampling, limits a model word choices to the most likely options that together cover a probability mass p, say the top 90 percent, then samples from just those. It adapts automatically: when the model is confident the pool is tiny, when it is unsure the pool widens. It is a common companion to temperature for controlling how focused or varied the output is.