Home › Key concepts › LoRA and efficient fine-tuning

Key concept · Walk (intermediate) · the same on every platform

LoRA and efficient fine-tuning

Fine-tune a big model by training a small add-on.

What it is

Instead of updating all of a model's billions of weights, LoRA trains small extra matrices that adjust it. It needs far less memory and time, and the add-on (adapter) is a small file you can swap.

Why it matters

It makes fine-tuning possible on a single GPU and cheap enough to try.

An example

A 7-billion-parameter model is tuned for claim summaries with a 50 MB adapter in two hours on one GPU.

The common mistake

Tuning on a few hundred messy examples and expecting magic.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Fine-tuningteach a model your style or task with examplesBedrock custom models / SageMakerFoundry fine-tuningVertex AI tuningMosaic AI fine-tuningHugging Face TRL or Unsloth

Related ideas

Where it shows up

Part of the Walk level on these paths: LLM Engineering (adapting models).