Home › Learn › Real-time analytics › Which route?

Real-time analytics · page 5 of 11

Which route: straight to ClickHouse, straight to BI, or through a platform

When to go straight to ClickHouse, straight to Power BI, or through a big-data platform, and the five reasons Kafka is worth adding.

↑ All of real-time analyticsReal time, explainedLambda vs KappaComponents, left to rightChange capture toolsWhich route?Eight designsDesigns on each cloudContracts and bindingsPush to the screenWhat it costsEstablished technologies

Four routes

Not every screen needs Kafka

ASourceRead replicaBI toolBSourceCDC toolClickHouseScreenCSourceCDCKafkaFlinkStore + lakeScreenDSourceCDC toolLakehousePower BI
Route A reads a replica; B goes straight into ClickHouse; C goes through Kafka; D goes through a lakehouse.
RouteUse it whenAvoid whenVerdict
A. The screen reads the sourceOne system, small data (up to tens of millions of rows), few users, simple totals. Always a read replica, never the main database.Joins across systems, heavy scans, many usersPursue for small operational boards
B. Straight into ClickHouseOne or two sources, ClickHouse is the only consumer, sub-second answers for many users, a small team. Reshaping happens in ClickHouse materialized views.Other systems need the same changes; heavy joins while data is movingPursue: the simplest real-time route
C. Through KafkaSeveral consumers, replay needed, enrichment or time windows needed, traffic spikes to absorb, many sourcesOne source and one target: Kafka only adds cost and workPursue with two or more consumers
D. Through a big-data platformA few minutes is fine; you need history, joins across many systems, machine learning, governance; the platform already existsUnder about 5 seconds with many users at oncePursue for near real time at scale
C and D togetherA live product screen and deep analyticsA tight budgetPursue as the mature end state

Is Kafka mandatory?

No. Five reasons to add it

You need Kafka, or a cloud log such as Event Hubs, Kinesis or Pub/Sub, if any of these is true:

  1. Two or more systems consume the same changes.
  2. You must be able to replay history through new logic.
  3. You enrich, join or count per time window while data moves.
  4. Microservices also react to the events, not only analytics.
  5. The source must be shielded from slow or failing consumers.

If none is true, use one change capture tool that writes straight to the target: ClickPipes into ClickHouse, Fabric Mirroring into OneLake, Datastream into BigQuery, Qlik Replicate into Snowflake, or Aurora zero-ETL into Redshift.

Power BI and friends

Where BI tools fit

Quick picks

Common cases

SituationPick
A Postgres app that needs a live analytics screenClickPipes or PeerDB straight into ClickHouse, then SSE to the screen. No Kafka.
Azure SQL into Power BI, a few minutes is fineFabric Mirroring into OneLake, then Power BI in Direct Lake mode. No Kafka.
Several databases, several consumers, real timeDebezium or Qlik into Kafka or Event Hubs, then Flink, then ClickHouse and the lakehouse
An Oracle- or mainframe-heavy enterpriseGoldenGate or Qlik Replicate into Kafka, then the platform
A small operations board on one systemGrafana or Power BI DirectQuery on a read replica

FAQ

Questions people ask

Is Kafka mandatory for real-time analytics?

No. One change capture tool writing straight to ClickHouse, a lakehouse or a warehouse is enough when one source feeds one target. Add Kafka when two or more systems consume the same changes, or when you need replay or processing in flight.

Can Power BI show real-time data?

Near real time, yes: automatic page refresh over DirectQuery, down to about a second on Fabric capacity. True push is not possible; streaming datasets are being retired in favour of Fabric Real-Time Intelligence.

When should data go straight into ClickHouse?

When one or two sources feed it, ClickHouse is the only consumer, and the reshaping is simple enough for ClickHouse materialized views.

Next

Where to go next

Next page: Eight real-time designs, without product names

↑ All of real-time analytics · Data platform architecture · Key concepts