Home › Key concepts › Choosing the right metric

Key concept · Walk (intermediate) · the same on every platform

Choosing the right metric

Accuracy is rarely the number that matters.

What it is

Precision is how many flagged cases were right; recall is how many real cases you caught. AUC summarises ranking quality. For numbers, MAE and RMSE measure the size of errors. Pick the metric that matches the business cost of each kind of mistake.

Why it matters

A fraud model that catches nothing can still be 99.9% accurate.

An example

Missing a fraud costs ₹50,000 and a false alarm costs a phone call, so the team optimises recall at a set precision.

The common mistake

Optimising a metric nobody in the business cares about.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Experiment trackingremember which run, data and settings gave which scoreSageMaker MLflowAzure ML (MLflow)Vertex AI ExperimentsMLflowMLflow

Related ideas

Where it shows up

Part of the Walk level on these paths: ML Engineering (learning basics).