Home › Key concepts › Data quality dimensions

Key concept · Crawl (beginner) · the same on every platform

Data quality dimensions

Six ways data goes wrong, and a check for each.

What it is

Completeness (are values missing?), uniqueness (are there duplicates?), validity (is the value allowed?), consistency (do two sources agree?), accuracy (is it true?) and timeliness (is it fresh?). Each can be written as an automatic check.

Why it matters

Checks at each layer stop bad data before a person or a model uses it.

An example

A check fails the silver load because 3% of orders have no customer id, instead of the gap appearing in a board report.

The common mistake

Checking quality only by eye, after someone complains.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Data qualitychecks that stop bad data before anyone uses itGlue Data QualityPurview data qualityDataplex data qualityPipeline expectationsGreat Expectations or Soda

Related ideas

Where it shows up

Part of the Crawl level on these paths: Data Engineering (trusting data) · ML Engineering (features and data).