Home › Key concepts › Change data capture (CDC)

Key concept · Walk (intermediate) · the same on every platform

Change data capture (CDC)

Copy only what changed in a database, by reading its change log.

What it is

Every database keeps a log of inserts, updates and deletes. Change data capture reads that log and sends each change downstream. Instead of copying a whole table again, you copy only the rows that changed, including deletes.

Why it matters

It is fast, light on the source database, and it catches deletes that a simple 'new rows since yesterday' query misses.

An example

A customer changes their address in the app. Within a minute the change lands in the bronze table as an update event.

The common mistake

Forgetting deletes: a customer closes their account, but the analytics copy keeps them forever.

Where you meet it on each platform

The idea stays the same; only the product name changes.

JobAWSAzureGoogle CloudDatabricksOpen source
Ingestioncopy data in from apps, databases and APIsAWS DMS / AppFlowData Factory copy / Fabric mirroringDatastreamLakeflow Connect / Auto Loaderdlt or Airbyte
Streaminghandle events as they happen, not once a nightKinesis / Amazon MSKEvent Hubs / Fabric EventstreamPub/Sub + DataflowStructured StreamingKafka or Redpanda

Related ideas

Where it shows up

Part of the Walk level on these paths: Data Engineering (moving data).