Home › Key concepts › Temperature and sampling
Key concept · Crawl (beginner) · the same on every platform
Temperature and sampling
How the model picks the next token from its options.
What it is
The model gives a probability for every possible next token. Temperature controls how adventurous the choice is: low picks the likeliest, high spreads the choice. Top-p limits the choice to the most likely few.
Why it matters
Low temperature suits extraction and code; higher suits brainstorming. The same prompt can behave very differently.
An example
Extracting a date at temperature 0 gives the same answer every time.
The common mistake
Expecting identical answers at high temperature, then filing it as a bug.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Model APIcall a large language model | Amazon Bedrock | Azure OpenAI in Microsoft Foundry | Gemini on Vertex AI | Foundation Model APIs | Ollama, or a free hosted open model |
Related ideas
Where it shows up
Part of the Crawl level on these paths: AI Engineering (working with models) · LLM Engineering (how models work).