Prompting & Inference
Beam Search
A decoding strategy that explores several candidate continuations at once and keeps the best overall sequence.
Instead of committing to the single most likely next token at each step, beam search keeps a handful of the most promising partial sequences alive in parallel and extends them, then returns the best complete one. It tends to produce more coherent, higher-probability output than greedy decoding, at the cost of more computation. It is especially common in translation and other tasks with a clearly correct target.
Reference: Wikipedia: Beam Search
Related terms
Inference
Actually running a trained model to get an answer, as opposed to training it in the first place.
Token
The small chunk of text a language model actually reads and writes, usually a word or part of a word.
Machine Translation
Automatically translating text or speech from one language to another, one of the oldest goals of AI.
Temperature
A setting that controls how random or predictable a model’s output is.