Speculative Decoding: How LLMs Generate Faster
Speculative decoding accelerates large language model inference by using a smaller, faster model to draft text that a larger, more powerful model then verifies in parallel, significantly reducing generation latency without compromising output quality.