Home › Key concepts › Tokens and the context window

Key concept · Crawl (beginner) · the same on every platform

Tokens and the context window

Models read and write in tokens, and can only see so many at once.

What it is

A token is a piece of a word, roughly three quarters of an English word. You pay per token in and out. The context window is the most tokens a model can consider in one call: the instructions, the history, the retrieved documents and its answer.

Why it matters

Cost, speed and quality all depend on what you put in the context and how much.

An example

A 30-page contract is about 15,000 tokens; sending it on every turn of a chat multiplies the bill.

The common mistake

Stuffing everything into the context 'just in case', making answers slower, dearer and often worse.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Model APIcall a large language modelAmazon BedrockAzure OpenAI in Microsoft FoundryGemini on Vertex AIFoundation Model APIsOllama, or a free hosted open model

Related ideas

Where it shows up

Part of the Crawl level on these paths: AI Engineering (working with models) · LLM Engineering (how models work).