Home › Key concepts › Temperature and sampling

Key concept · Crawl (beginner) · the same on every platform

Temperature and sampling

How the model picks the next token from its options.

What it is

The model gives a probability for every possible next token. Temperature controls how adventurous the choice is: low picks the likeliest, high spreads the choice. Top-p limits the choice to the most likely few.

Why it matters

Low temperature suits extraction and code; higher suits brainstorming. The same prompt can behave very differently.

An example

Extracting a date at temperature 0 gives the same answer every time.

The common mistake

Expecting identical answers at high temperature, then filing it as a bug.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Model APIcall a large language modelAmazon BedrockAzure OpenAI in Microsoft FoundryGemini on Vertex AIFoundation Model APIsOllama, or a free hosted open model

Related ideas

Where it shows up

Part of the Crawl level on these paths: AI Engineering (working with models) · LLM Engineering (how models work).