Home › Key concepts › Quantisation
Key concept · Walk (intermediate) · the same on every platform
Quantisation
Store a model's numbers with fewer bits so it runs smaller and faster.
What it is
Model weights are usually 16-bit numbers. Quantisation stores them in 8 or 4 bits. The model uses a half or a quarter of the memory and runs faster, with a small loss of quality you should measure.
Why it matters
It lets good open models run on cheaper hardware, even laptops.
An example
A model that needs 28 GB in 16-bit fits in about 8 GB at 4-bit.
The common mistake
Quantising and never checking quality on your own task.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Model hosting on GPUsrun open-weight models yourself, fast and cheap per token | SageMaker endpoints / EKS + vLLM | Foundry managed compute / AKS + vLLM | Vertex AI endpoints / GKE + vLLM | Provisioned throughput serving | vLLM |
Related ideas
Where it shows up
Part of the Walk level on these paths: LLM Engineering (serving models).