Home › Key concepts › Freshness and SLAs
Key concept · Walk (intermediate) · the same on every platform
Freshness and SLAs
Promise how up to date data will be, then measure it.
What it is
Freshness is the time since the data was last updated. An SLA (service level agreement) is the promise, such as 'sales are less than 2 hours old by 9 a.m.'. You measure it and alert before users notice.
See it happen: Problem Lab: a source goes dark, and a freshness SLO that alerts once (simulation), recorded from a real run.
Why it matters
Users trust data that is on time. A late table is often worse than a wrong one, because nobody knows.
An example
An alert fires at 08:30 because the orders table is 5 hours old, before the 9 a.m. meeting.
The common mistake
Promising 'real time' without saying what that means or measuring it.
Where you meet it on each platform
The idea stays the same; only the product name changes.
| Job | AWS | Azure | Google Cloud | Databricks | Open source |
|---|---|---|---|---|---|
| Orchestrationrun the steps in order, on time, and retry the failures | MWAA (Airflow) / Step Functions | Data Factory / Fabric pipelines | Cloud Composer (Airflow) | Lakeflow Jobs | Airflow or Dagster |
| Monitoringlogs, metrics and traces, so you see trouble first | CloudWatch + X-Ray | Azure Monitor + App Insights | Cloud Logging + Trace | System tables + MLflow tracing | OpenTelemetry + Grafana |
Related ideas
Where it shows up
Part of the Walk level on these paths: Data Engineering (running data).