Home › Key concepts › Agree the measure of success first

Key concept · Crawl (beginner) · the same on every platform

Agree the measure of success first

Decide with the customer how you will know it worked, before you build.

What it is

Pick one or two numbers that matter to them, such as minutes per claim, error rate or cost per case, measure today's value, and agree the target. It turns a demo into an outcome.

Why it matters

Without it, success is opinion, and projects drift.

An example

Today: 40 minutes per claim, 6% re-work. Target in 8 weeks: under 15 minutes, re-work no higher.

The common mistake

Measuring only what is easy, such as number of chats.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Evaluationprove it works on cases you wrote down, every releaseBedrock evaluationsFoundry evaluationsVertex AI Gen AI evaluationMLflow evaluationEval Harness Framework or promptfoo
Costsee what each job and query costs, then cut itCost Explorer + CURCost ManagementBilling export to BigQueryBilling system tablesQuery plans and run times

Related ideas

Where it shows up

Part of the Crawl level on these paths: Forward Deployed Engineering (understand).