Home › Key concepts › LoRA and efficient fine-tuning
Key concept · Walk (intermediate) · the same on every platform
LoRA and efficient fine-tuning
Fine-tune a big model by training a small add-on.
What it is
Instead of updating all of a model's billions of weights, LoRA trains small extra matrices that adjust it. It needs far less memory and time, and the add-on (adapter) is a small file you can swap.
Why it matters
It makes fine-tuning possible on a single GPU and cheap enough to try.
An example
A 7-billion-parameter model is tuned for claim summaries with a 50 MB adapter in two hours on one GPU.
The common mistake
Tuning on a few hundred messy examples and expecting magic.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Fine-tuningteach a model your style or task with examples | Bedrock custom models / SageMaker | Foundry fine-tuning | Vertex AI tuning | Mosaic AI fine-tuning | Hugging Face TRL or Unsloth |
Related ideas
Where it shows up
Part of the Walk level on these paths: LLM Engineering (adapting models).