Home › Learn › Real-time analytics › Real time, explained
Real-time analytics · page 1 of 11
Real time vs streaming vs near real time
Real-time analytics is a result; streaming is one way to move data. Real time (under a second) and near real time (seconds to minutes) need different designs and budgets.
In plain words
The result vs the way of moving data
Streaming is a way of moving and processing data: one event at a time, all the time. Real-time analytics is a result: questions about very fresh data, answered in under a second, for many people at once. Streaming is usually how the data gets there, but you also need a store built for fast questions and a screen that receives updates.
You can have either without the other. A stream that feeds a report printed every night is still streaming. And you can send raw events straight into a fast analytics database, such as ClickHouse, with no stream-processing engine at all, and still get real-time answers.
| Streaming | Real-time analytics | |
|---|---|---|
| What it is | A way of moving data, event by event, continuously | A result: fresh answers in under a second, for many users |
| Question it answers | How does data move? | How fresh and how fast are the answers? |
| Can it exist alone? | Yes: streaming into a nightly report | Yes: raw events straight into ClickHouse, no processing engine |
Freshness
Real time and near real time are different
| Tier | Delay, end to end | Who acts on it | Examples | Typical design |
|---|---|---|---|---|
| Hard real time | Microseconds to milliseconds, guaranteed | Machines | Factory control, car brakes, high-frequency trading | Embedded systems and special hardware. Not Kafka or analytics platforms; out of scope here. |
| Real time | Under about 1 second | Machines, or a person who must react right now | Blocking a fraudulent payment, ad bidding, a live operations room | Event log, stream processor, fast analytics store, push to the screen |
| Near real time | About 5 seconds to 15 minutes | People watching dashboards | Today's sales, stock levels, call-centre boards | Small batches into a warehouse or lakehouse; the dashboard refreshes every 30 to 60 seconds |
| Batch | Hours to a day | People planning | Finance reports, monthly cost reports | A nightly job |
The one question
Who acts, and how fast can they act?
Ask what decision changes, who makes it, and how quickly they can act on it. A person watching a dashboard cannot act on a change that is half a second old, so near real time is almost always enough for people. A program that blocks a payment can act in milliseconds, so it needs real time.
Most requests for a "real-time dashboard" turn out to be near real time once you ask this question. That matters, because real time usually costs three to ten times as much as the batch version of the same report (see what it costs).
The cost line: near real time stays cheap only while compute can switch off between batches. Below about a five-minute interval, warehouse compute never gets to pause, so you pay for always-on compute and still have a batch design. At that point a proper streaming design is often cheaper.
FAQ
Questions people ask
Is real-time analytics the same as streaming?
No. Streaming is a way of moving data, event by event. Real-time analytics is the result: fresh answers in under a second for many users. Streaming is usually the transport, but you also need a fast analytics store and a screen that receives updates.
What is the difference between real time and near real time?
Real time is under about one second end to end and is needed when a machine acts on the data. Near real time is seconds to about fifteen minutes and is enough when people watch a dashboard. Near real time is usually much cheaper.
Is Kafka real time?
Kafka itself moves events in milliseconds, but it is soft real time: fast on average, with no guaranteed deadline. Hard real-time systems, such as brakes, use special hardware, not Kafka.
Next
Where to go next
Next page: Lambda vs Kappa, drawn simply
↑ All of real-time analytics · Data platform architecture · Key concepts