Home › Learn › Real-time analytics › Eight designs
Real-time analytics · page 6 of 11
Eight real-time designs, without product names
Eight abstract real-time analytics designs built from nine roles, with no product names: when to use each, delay, cost, effort and a verdict.
The building blocks
Nine roles, no product names
Every design on this page uses the same nine roles. A design is just a choice of which roles to use. Products come later, on the physical designs page.
| # | Role | What it does |
|---|---|---|
| R1 | Source | the app and its database |
| R2 | Capture | turns changes into events |
| R3 | Event log | keeps events, allows replay |
| R4 | Process | clean, join, count windows |
| R5 | Serve | fast answers, many users |
| R6 | History | cheap, full history |
| R7 | API | metrics, security |
| R8 | Push | sends changes to screens |
| R9 | Screen | dashboard or app |
At a glance
Eight designs
| Design | Typical delay | Small monthly cost | Running effort | Verdict |
|---|---|---|---|---|
| L1 Direct query | Seconds (replica lag) | Lowest | Low | Pursue for one system, small data |
| L2 Direct capture into the serving store | 1 to 5 s | $500–2k a month | Low to medium | Pursue: the simplest real-time design |
| L3 Log-centred streaming (Kappa) | Under 1 to 3 s | $2–6k a month | High | Pursue with several consumers |
| L4 Lakehouse, near real time | 1 to 15 min | $0.5–3k a month | Medium | Pursue when people are the ones acting |
| L5 Kappa plus lakehouse | Under 1 s live, plus full history | $5k+ a month | High | Pursue as the end state |
| L6 Streaming database | Under 1 s | $1–4k a month | Medium | Pursue for continuous joins plus push |
| L7 HTAP: one database for both | Seconds | Varies | Low | Pursue only for one system on a capable database |
| L8 Lambda | Mixed | High | Very high | Drop: legacy |
Designs L1, L2, L6 and L7 skip the log; they are fine when one source feeds one consumer. L3 and L5 put the log at the centre, which you need once there are several consumers or a need to replay. L4 gives up speed for history and breadth. Costs are rough; see what it costs.
Each design
The eight, one by one
L1 · Direct query
The BI tool reads a replica of the source database.
Pursue for one system, small data
Use it when: One system, small data, few users. Avoid when: Joins across systems, many users. Products for L1
L2 · Direct capture into the serving store
A change capture tool writes straight into ClickHouse or similar; reshaping happens inside the store.
Pursue: the simplest real-time design
Use it when: One or two sources, one consumer, sub-second answers. Avoid when: Other systems need the same changes. Products for L2
L3 · Log-centred streaming (Kappa)
Every change flows through an event log; a processor builds curated topics; a fast store serves; changes are pushed.
Pursue with several consumers
Use it when: Several consumers, replay, enrichment in flight. Avoid when: One source, one target. Products for L3
L4 · Lakehouse, near real time
Change capture into Databricks, Fabric or Snowflake in small batches; the warehouse serves the dashboard.
Pursue when people are the ones acting
Use it when: A few minutes is fine; history and joins across many systems. Avoid when: Sub-second answers for many users. Products for L4
L5 · Kappa plus lakehouse
One log feeds a live store and a full history; the mature enterprise design.
Pursue as the end state
Use it when: Live screens and deep analytics together. Avoid when: A tight budget or a small team. Products for L5
L6 · Streaming database
One engine keeps query results up to date as data changes, and pushes the changes.
Pursue for continuous joins plus push
Use it when: A live screen built on joins that must stay up to date. Avoid when: You need wide tool support; the products are still maturing. Products for L6
L7 · HTAP: one database for both
The transaction database also answers analytical questions, using a column engine inside it.
Pursue only for one system on a capable database
Use it when: One system whose database has a column engine. Avoid when: Several sources; analytics would slow the app. Products for L7
L8 · Lambda
A batch path and a speed path, merged in a serving layer.
Drop: legacy
Use it when: Only when you inherit it. Avoid when: Any new build. Products for L8
Choosing
Which one?
| If | Then |
|---|---|
| One system, small data, few users | L1 |
| One or two sources, sub-second answers, only one consumer | L2 |
| A few minutes is fine; history and joins across many systems | L4 |
| Several consumers, replay, enrichment in flight | L3 |
| All of that, plus live screens and deep analytics | L5, usually reached by growing from L2 or L4 |
| A live screen built on joins that stay up to date, with push | L6 |
A sensible path is to start with L2 or L4 and add the log when a second consumer appears. If each role sits behind its own interface, with a contract per topic and the product named only in a binding, moving between designs means swapping products, not rewriting. See contracts and bindings.
FAQ
Questions people ask
What are the main real-time analytics architectures?
Direct query on a replica, direct capture into a serving store, log-centred streaming (Kappa), lakehouse near real time, Kappa plus lakehouse, a streaming database, HTAP, and the older Lambda.
Which real-time architecture should I start with?
Usually direct capture into a serving store such as ClickHouse (real time) or a lakehouse with small batches (near real time). Add an event log such as Kafka when a second consumer appears.
What is HTAP?
Hybrid transactional and analytical processing: one database handles both the app's transactions and analytical queries, using a column engine inside it. Examples: SingleStore, TiDB, AlloyDB, SQL Server columnstore.
Next
Where to go next
Next page: The eight designs on Azure, AWS, Google Cloud and open source
↑ All of real-time analytics · Data platform architecture · Key concepts