Definition
What is Top-p sampling?
Top-p sampling is Top-p sampling, also called nucleus sampling, is a setting that limits an AI model's next-word choice to the smallest group of likely tokens whose probabilities add up to p, such as 95%.
What it is
When a language model writes, it assigns a probability to every possible next token. Top-p sampling decides which of those tokens are even allowed to be picked.
Here is how it works. The model sorts the tokens from most to least likely. It then adds up their probabilities from the top until the total reaches a threshold called p. If p is 0.95, it keeps the shortest list of tokens that together cover 95% of the probability. Everything else is thrown away, and the model picks from what remains.
Imagine a restaurant menu. Top-p trims the menu to the dishes most people actually order and removes the odd ones that would rarely be chosen, then you pick from what is left.
Why the list changes size
The key feature is that the list is not a fixed length. If the model is very sure, say the next word is almost certainly "York" after "New," one or two tokens may already cover 95%, so the list is tiny. If the model is unsure, many tokens may be needed to reach 95%, so the list is longer. The cutoff adapts to how confident the model is at each step.
This is different from top-k sampling, which always keeps a fixed number of tokens, such as the top 40, regardless of how the probabilities look.
How it differs from temperature
Temperature reshapes the odds across all tokens. Top-p removes the unlikely tail entirely. They are often used together. A common pattern is a moderate temperature with a top-p near 0.9 to 0.95, which keeps variety while cutting the rare words that tend to cause nonsense.
Many providers advise changing one of the two at a time rather than both, because their effects overlap. Some newer reasoning models do not let you change these settings at all.
When to use it
Lowering top-p makes output safer and more predictable. Raising it toward 1.0 allows the full range of choices. For tasks that need exact output, such as code or data extraction, a low temperature is usually the simpler tool. For creative work, a higher top-p with a modest temperature gives variety without as many strange word choices.
Like temperature, top-p changes only how the final word is picked. It does not change what the model knows, and it will not stop a hallucination on its own.
Questions people ask
What is the difference between top-p and temperature?
Temperature changes how spread out the odds are across all tokens, while top-p cuts off the unlikely tail so only the most probable tokens can be chosen.
What is a good top-p value?
Values around 0.9 to 0.95 are common. Lower is more predictable, and 1.0 allows every token.