Home › Key concepts › Agree the measure of success first
Key concept · Crawl (beginner) · the same on every platform
Agree the measure of success first
Decide with the customer how you will know it worked, before you build.
What it is
Pick one or two numbers that matter to them, such as minutes per claim, error rate or cost per case, measure today's value, and agree the target. It turns a demo into an outcome.
Why it matters
Without it, success is opinion, and projects drift.
An example
Today: 40 minutes per claim, 6% re-work. Target in 8 weeks: under 15 minutes, re-work no higher.
The common mistake
Measuring only what is easy, such as number of chats.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Evaluationprove it works on cases you wrote down, every release | Bedrock evaluations | Foundry evaluations | Vertex AI Gen AI evaluation | MLflow evaluation | Eval Harness Framework or promptfoo |
| Costsee what each job and query costs, then cut it | Cost Explorer + CUR | Cost Management | Billing export to BigQuery | Billing system tables | Query plans and run times |
Related ideas
Where it shows up
Part of the Crawl level on these paths: Forward Deployed Engineering (understand).