Prompting & Inference

Beam Search

A decoding strategy that explores several candidate continuations at once and keeps the best overall sequence.

Instead of committing to the single most likely next token at each step, beam search keeps a handful of the most promising partial sequences alive in parallel and extends them, then returns the best complete one. It tends to produce more coherent, higher-probability output than greedy decoding, at the cost of more computation. It is especially common in translation and other tasks with a clearly correct target.

Reference: Wikipedia: Beam Search