Home › Learn › Real-time analytics › Designs on each cloud
Real-time analytics · page 7 of 11
The eight designs on Azure, AWS, Google Cloud and open source
The eight real-time analytics designs with products for each role on Azure, Fabric, AWS, Google Cloud, Databricks, Snowflake and open source.
Concept vs product
Same roles, different products
The eight designs name no product. This page gives each role a product, on each cloud. The design is the logic; the product is the language you write it in. Changing cloud means a new set of products for the same roles, not a new design.
Product names change often; check the current docs before you build.
Design by design
Products for each role
L1 · Direct query
The BI tool reads a replica of the source database. The abstract design
| Role | Azure | AWS | Open source |
|---|---|---|---|
| Source and replica | Azure SQL or Hyperscale named replica, Postgres Flexible read replica | RDS or Aurora read replica | Postgres or MySQL streaming replica |
| Screen | Power BI DirectQuery with automatic page refresh | QuickSight, Grafana | Grafana, Metabase, Superset |
L2 · Direct capture into the serving store
A change capture tool writes straight into ClickHouse or similar; reshaping happens inside the store. The abstract design
| Role | Azure | AWS | Google Cloud | Open source |
|---|---|---|---|---|
| Capture | Fabric Eventstream CDC connectors | ClickPipes, or Aurora zero-ETL | Datastream | PeerDB, ClickPipes, Estuary |
| Serve | Fabric Eventhouse, or ClickHouse Cloud on Azure | ClickHouse Cloud, or Redshift via zero-ETL | BigQuery with continuous queries | ClickHouse |
| Reshaping | KQL update policies | ClickHouse materialized views | BigQuery SQL | ClickHouse materialized views |
| Screen and push | Real-Time Dashboard, Power BI; Activator for alerts | Grafana, or a small SSE service | Looker | Grafana, or a service that polls the store and pushes only changes |
L3 · Log-centred streaming (Kappa)
Every change flows through an event log; a processor builds curated topics; a fast store serves; changes are pushed. The abstract design
| Role | Azure | AWS | Open source and neutral |
|---|---|---|---|
| Capture | Debezium on Container Apps or AKS, writing to Event Hubs; or Qlik | DMS, or Debezium on MSK Connect | Debezium |
| Event log and schemas | Event Hubs Premium with its schema registry | MSK with Glue Schema Registry | Confluent Cloud with Schema Registry, or Redpanda |
| Process | Stream Analytics, Databricks Structured Streaming, or Confluent Flink on Azure | Managed Service for Apache Flink | Confluent Flink, or Flink you run |
| Serve | Eventhouse or Azure Data Explorer, or ClickHouse Cloud | ClickHouse Cloud, Pinot (StarTree), OpenSearch | ClickHouse, Pinot, Druid |
| API and push | API on Container Apps with Azure Web PubSub or SignalR | API Gateway WebSocket APIs, or AppSync subscriptions | A Node or Go gateway that reads a results topic and sends SSE |
| Screen | React | React | React |
L4 · Lakehouse, near real time
Change capture into Databricks, Fabric or Snowflake in small batches; the warehouse serves the dashboard. The abstract design
| Role | Fabric | Databricks | Snowflake | AWS | Google Cloud |
|---|---|---|---|---|---|
| Capture | Fabric Mirroring | Lakeflow Connect | Openflow, Snowpipe Streaming | DMS, or Aurora zero-ETL | Datastream |
| Store and reshape | OneLake Delta tables, materialized lake views | Lakeflow Declarative Pipelines (streaming tables) | Dynamic Tables with a target lag of about a minute | S3 Tables (Iceberg) or Redshift | BigQuery |
| Screen | Power BI in Direct Lake mode | Power BI, AI/BI dashboards | Power BI, Sigma | QuickSight | Looker |
L5 · Kappa plus lakehouse
One log feeds a live store and a full history; the mature enterprise design. The abstract design
| Role | Azure with Databricks | Azure, all Fabric | AWS | Open source and neutral |
|---|---|---|---|---|
| Capture | Debezium or Qlik | Eventstream CDC | DMS or Debezium | Debezium |
| Event log | Event Hubs | Eventstream | MSK | Confluent Kafka |
| Process | Databricks Structured Streaming | Eventstream transforms, KQL | Managed Flink | Flink |
| Live serve | Eventhouse, Azure Data Explorer, or ClickHouse | Eventhouse | ClickHouse, Pinot | ClickHouse |
| History | Delta on ADLS (bronze, silver, gold) | Eventhouse OneLake availability (the same data as Delta) | Iceberg on S3 Tables | Iceberg through Confluent Tableflow, read by Trino, Databricks or Snowflake |
| Screen | Power BI for analytics, your own SSE app for live | Real-Time Dashboard, Power BI | QuickSight and your own app | Superset and your own app |
L6 · Streaming database
One engine keeps query results up to date as data changes, and pushes the changes. The abstract design
| Role | Options |
|---|---|
| Capture | The engine's own Postgres or MySQL CDC, or reading from Kafka |
| Engine | RisingWave (its cloud, or run on AKS or EKS), Materialize |
| Push | SUBSCRIBE in the engine feeds a small SSE or WebSocket gateway |
| On Azure | No native service does this. The closest is Eventhouse materialized views plus Activator; otherwise the vendor's cloud or AKS. |
L7 · HTAP: one database for both
The transaction database also answers analytical questions, using a column engine inside it. The abstract design
| Database | How it does analytics | Push |
|---|---|---|
| Azure SQL, SQL Server | Columnstore indexes on the transaction tables | Power BI page refresh |
| AlloyDB for Postgres | A built-in column engine | None built in |
| SingleStore, TiDB (TiFlash) | Row and column storage in one database | None built in |
| Oracle Database In-Memory | An in-memory column store | None built in |
| Postgres with pg_duckdb or pg_mooncake | The DuckDB engine inside Postgres | Supabase Realtime, or LISTEN/NOTIFY |
| Cosmos DB | Fabric Mirroring (which replaces Synapse Link) | Change feed |
L8 · Lambda
A batch path and a speed path, merged in a serving layer. The abstract design
| Role | What it used to be |
|---|---|
| Batch path | Hadoop or Spark batch jobs |
| Speed path | Storm or Spark Streaming |
| Serving | HBase or Cassandra |
| Verdict | Drop; for recognising older systems only |
FAQ
Questions people ask
What is the Azure architecture for real-time analytics?
For real time: Debezium or Fabric Eventstream CDC, Event Hubs, Stream Analytics or Databricks, Fabric Eventhouse or Azure Data Explorer, and SignalR or Web PubSub for push. For near real time: Fabric Mirroring into OneLake and Power BI in Direct Lake mode.
What is the AWS architecture for real-time analytics?
DMS or Debezium, MSK or Kinesis, Managed Service for Apache Flink, ClickHouse Cloud or Pinot, and API Gateway WebSocket APIs. For near real time, Aurora zero-ETL into Redshift.
Can the same design run on any cloud?
Yes, if every role sits behind its own interface. Only the binding, which product plays each role, changes.
Next
Where to go next
Next page: Contracts and bindings at every hop
↑ All of real-time analytics · Data platform architecture · Key concepts