Definition
Greedy decoding is Greedy decoding is a way of generating text in which an AI model always picks the single most likely next token at each step, which makes its output predictable but less varied.
A language model does not write a whole answer at once. It produces one token at a time, and at each step it scores every possible next token. A decoding method is the rule for choosing among those scores. Greedy decoding uses the simplest rule: take the highest-scoring token, every time.
Say a model has read "The capital of France is". It assigns the highest probability to " Paris", so greedy decoding writes " Paris", adds it to the text, and repeats the process for the next token. It never looks ahead and never reconsiders.
Because there is no randomness, the same prompt gives the same answer each time, provided the model and software are the same. Small numerical differences in hardware can still cause rare variation.
Most chat products do not decode greedily. They sample, which means they sometimes pick a lower-ranked token so answers feel more natural and varied. Two settings control this:
Greedy decoding is the extreme case where the pool holds exactly one token.
Greedy decoding is fast, simple and repeatable, which makes it useful for tests, structured output and anything where you need the same result twice. It also makes results easier to verify and compare.
The weakness is that the best token now does not guarantee the best answer overall. Greedy output can drift into repetitive loops, and it can miss a better sentence that begins with a slightly less likely word. Creative writing usually suffers.
If you need consistent results, such as running an evals suite, set temperature to zero or choose a greedy option when the provider offers one. If you want variety, leave sampling on. Some verification and testing techniques also rely on greedy choices because they are deterministic and therefore checkable.
They are very close in practice. Both pick the most likely token, though providers may implement temperature zero slightly differently.
It can produce repetitive or dull text and can miss better answers that start with a less likely word.
Anthropic and university researchers released FlashProve, an open-source system that cryptographically proves a 400B-parameter model produced a given token, at about 72 times the time of the prefill. It's slow and narrow, and it's a working answer to "did they swap my model?"