Temperature is a parameter that controls how much randomness a language model injects into its next-token predictions, where a low value makes output deterministic and repetitive and a high value makes it more varied and occasionally incoherent, tuned per task rather than left at a single default.
How does temperature actually change a model’s output?
At each step, an LLM produces a probability distribution over the next possible token, and temperature reshapes that distribution before a token is sampled. Near zero, the model almost always picks the single highest-probability token, so inference becomes close to deterministic and repeatable across calls. Push it higher and lower-probability tokens get a real chance of being picked, which is what produces more varied phrasing and, past a point, output that drifts off topic or stops making sense. There is no universally correct setting, only the right one for what the output needs to do.
What temperature should a production system actually use?
Near zero for anything that must be consistent or parsed by a program: tool arguments, structured output, classification labels. Higher for creative drafting, brainstorming, or varied phrasing across many generations. Temperature is unrelated to how a model reasons through a problem; it only shapes sampling at the output layer, which is why some reasoning models restrict or ignore it entirely during their internal reasoning steps.
This entry was drafted with AI assistance.
Frequently asked questions
Does a higher temperature make a model smarter?
No. Temperature only changes how randomly the next token is sampled, not how well the model reasons. A higher value adds variety and sometimes creativity, but it can also introduce errors a lower setting would have avoided.
What temperature should I use for tool calling?
Close to zero. Tool arguments and structured output need to be consistent and parseable, and a low temperature makes the model's token choices close to deterministic, which is what a program calling that output depends on.
No advertising or tracking cookies, and our visitor counts are anonymous. The Cal.com booking widget loads only if you allow it. Privacy Policy.
The page itself, anything our host sets to serve and secure it, and the anonymous visitor count. Always on, and none of it stores anything on your device.
The Cal.com booking widget. Left off, a booking link opens the booking page instead of a popup, so you can still book a call.