What it is
A language model does not pick its next word by looking something up. It scores every possible next token, turns the scores into probabilities, and then picks one. Temperature is the dial that decides how strictly the model follows those probabilities.
A low temperature, close to 0, makes the model nearly always pick the most likely token. Answers become focused and consistent, and asking the same question twice usually gives the same reply. A higher temperature, around 1 or above, flattens the odds, so less likely tokens get picked more often. Answers become more varied, more creative and more likely to wander.
Think of a musician playing from sheet music. At low temperature they play the notes as written. At high temperature they improvise.
How it works
Before the probabilities are computed, the model's raw scores are divided by the temperature. A small temperature makes the gaps between scores larger, so the top choice dominates. A large temperature shrinks the gaps, so many choices stay in play. At exactly 0, most services simply take the single top token every time.
When to change it
Use a low setting when you want reliable output: extracting data, writing code, following a format, or running an agent where a surprise could break something. Use a higher setting for brainstorming, naming ideas and writing where you want variety.
Higher temperature does not make a model smarter or more imaginative in any deep way. It only makes it less predictable, and it can raise the chance of a hallucination or a broken answer. Low temperature does not guarantee truth either. A model can repeat the same wrong answer with perfect consistency.
Providers set their own defaults and sometimes recommend specific values for specific tasks, and some reasoning models fix or limit the setting. Check the model's documentation before you tune it.
Temperature is usually used alongside top-p sampling, which limits the pool of tokens the model may choose from. Together they are called sampling settings. Both act only on the final choice of word, not on what the model knows, and neither changes the model's parameters.