Home › Key concepts › Choosing the right metric
Key concept · Walk (intermediate) · the same on every platform
Choosing the right metric
Accuracy is rarely the number that matters.
What it is
Precision is how many flagged cases were right; recall is how many real cases you caught. AUC summarises ranking quality. For numbers, MAE and RMSE measure the size of errors. Pick the metric that matches the business cost of each kind of mistake.
Why it matters
A fraud model that catches nothing can still be 99.9% accurate.
An example
Missing a fraud costs ₹50,000 and a false alarm costs a phone call, so the team optimises recall at a set precision.
The common mistake
Optimising a metric nobody in the business cares about.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Experiment trackingremember which run, data and settings gave which score | SageMaker MLflow | Azure ML (MLflow) | Vertex AI Experiments | MLflow | MLflow |
Related ideas
Where it shows up
Part of the Walk level on these paths: ML Engineering (learning basics).