Temperature is a parameter that controls how much randomness a language model injects into its next-token predictions, where a low value makes output deterministic and repetitive and a high value makes it more varied and occasionally incoherent, tuned per task rather than left at a single default.
How does temperature actually change a model’s output?
At each step, an LLM produces a probability distribution over the next possible token, and temperature reshapes that distribution before a token is sampled. Near zero, the model almost always picks the single highest-probability token, so inference becomes close to deterministic and repeatable across calls. Push it higher and lower-probability tokens get a real chance of being picked, which is what produces more varied phrasing and, past a point, output that drifts off topic or stops making sense. There is no universally correct setting, only the right one for what the output needs to do.
What temperature should a production system actually use?
Near zero for anything that must be consistent or parsed by a program: tool arguments, structured output, classification labels. Higher for creative drafting, brainstorming, or varied phrasing across many generations. Temperature is unrelated to how a model reasons through a problem; it only shapes sampling at the output layer, which is why some reasoning models restrict or ignore it entirely during their internal reasoning steps.
Frequently asked questions
Does a higher temperature make a model smarter?
No. Temperature only changes how randomly the next token is sampled, not how well the model reasons. A higher value adds variety and sometimes creativity, but it can also introduce errors a lower setting would have avoided.
What temperature should I use for tool calling?
Close to zero. Tool arguments and structured output need to be consistent and parseable, and a low temperature makes the model's token choices close to deterministic, which is what a program calling that output depends on.