Home › Learn › Real-time analytics › Which route?
Real-time analytics · page 5 of 11
Which route: straight to ClickHouse, straight to BI, or through a platform
When to go straight to ClickHouse, straight to Power BI, or through a big-data platform, and the five reasons Kafka is worth adding.
Four routes
Not every screen needs Kafka
| Route | Use it when | Avoid when | Verdict |
|---|---|---|---|
| A. The screen reads the source | One system, small data (up to tens of millions of rows), few users, simple totals. Always a read replica, never the main database. | Joins across systems, heavy scans, many users | Pursue for small operational boards |
| B. Straight into ClickHouse | One or two sources, ClickHouse is the only consumer, sub-second answers for many users, a small team. Reshaping happens in ClickHouse materialized views. | Other systems need the same changes; heavy joins while data is moving | Pursue: the simplest real-time route |
| C. Through Kafka | Several consumers, replay needed, enrichment or time windows needed, traffic spikes to absorb, many sources | One source and one target: Kafka only adds cost and work | Pursue with two or more consumers |
| D. Through a big-data platform | A few minutes is fine; you need history, joins across many systems, machine learning, governance; the platform already exists | Under about 5 seconds with many users at once | Pursue for near real time at scale |
| C and D together | A live product screen and deep analytics | A tight budget | Pursue as the mature end state |
Is Kafka mandatory?
No. Five reasons to add it
You need Kafka, or a cloud log such as Event Hubs, Kinesis or Pub/Sub, if any of these is true:
- Two or more systems consume the same changes.
- You must be able to replay history through new logic.
- You enrich, join or count per time window while data moves.
- Microservices also react to the events, not only analytics.
- The source must be shielded from slow or failing consumers.
If none is true, use one change capture tool that writes straight to the target: ClickPipes into ClickHouse, Fabric Mirroring into OneLake, Datastream into BigQuery, Qlik Replicate into Snowflake, or Aurora zero-ETL into Redshift.
Power BI and friends
Where BI tools fit
- Power BI does not push sub-second updates. Its best is automatic page refresh over DirectQuery, down to about a second on Fabric or Premium capacity, with change detection.
- It works well over Fabric Eventhouse or Azure Data Explorer, over ClickHouse (there is an official connector), and over Databricks SQL. That is near real time to about a second: fine for people.
- Power BI real-time streaming datasets are being retired in favour of Fabric Real-Time Intelligence. Do not build on them.
- For live operations boards, Grafana over ClickHouse or Azure Data Explorer refreshes every one to five seconds and costs less.
- For true push on a customer-facing screen, build your own screen with SSE or WebSocket. See push to the screen.
Quick picks
Common cases
| Situation | Pick |
|---|---|
| A Postgres app that needs a live analytics screen | ClickPipes or PeerDB straight into ClickHouse, then SSE to the screen. No Kafka. |
| Azure SQL into Power BI, a few minutes is fine | Fabric Mirroring into OneLake, then Power BI in Direct Lake mode. No Kafka. |
| Several databases, several consumers, real time | Debezium or Qlik into Kafka or Event Hubs, then Flink, then ClickHouse and the lakehouse |
| An Oracle- or mainframe-heavy enterprise | GoldenGate or Qlik Replicate into Kafka, then the platform |
| A small operations board on one system | Grafana or Power BI DirectQuery on a read replica |
FAQ
Questions people ask
Is Kafka mandatory for real-time analytics?
No. One change capture tool writing straight to ClickHouse, a lakehouse or a warehouse is enough when one source feeds one target. Add Kafka when two or more systems consume the same changes, or when you need replay or processing in flight.
Can Power BI show real-time data?
Near real time, yes: automatic page refresh over DirectQuery, down to about a second on Fabric capacity. True push is not possible; streaming datasets are being retired in favour of Fabric Real-Time Intelligence.
When should data go straight into ClickHouse?
When one or two sources feed it, ClickHouse is the only consumer, and the reshaping is simple enough for ClickHouse materialized views.
Next
Where to go next
Next page: Eight real-time designs, without product names
↑ All of real-time analytics · Data platform architecture · Key concepts