Home › Learn › Real-time analytics › Established technologies
Real-time analytics · page 11 of 11
Which real-time technologies are established
Which real-time analytics technologies are established, growing or fading, layer by layer: CDC, event logs, contracts, processing, serving, history and push.
At a glance
Safe, growing, or drop
A technology counts as established here when it is widely used in production, offered by more than one vendor or cloud, and easy to hire for. Growing ones are fine when you have a reason. The last column is for recognising older systems, not for new builds.
Layer by layer
The full list
| Layer | Established (safe) | Growing (fine with a reason) | Legacy or fading (drop) |
|---|---|---|---|
| Change capture | Debezium, GoldenGate, AWS DMS, native change feeds (Cosmos DB, SQL Server CDC) | Estuary, Flink CDC | Trigger-based capture, polling timestamp columns |
| Event log | Apache Kafka (no ZooKeeper since 4.0), Kinesis, Event Hubs, Pub/Sub | Redpanda; Kafka on object storage (WarpStream, AutoMQ, diskless Kafka) | RabbitMQ or SQS for analytics (no replay); Pulsar is losing momentum |
| Contracts | Avro or Protobuf with a schema registry, CloudEvents | AsyncAPI 3, ODCS data contracts | JSON with no schema |
| Processing | Apache Flink, Spark Structured Streaming, Kafka Streams | RisingWave, Materialize, Spark real-time mode | Storm, Samza, Spark DStreams, ksqlDB (Confluent moved to Flink) |
| Serving | ClickHouse, Apache Pinot, Apache Druid, Elasticsearch or OpenSearch, Redis, Azure Data Explorer | StarRocks, Fabric Eventhouse | Lambda-style batch-plus-speed serving layers |
| History | Delta, Iceberg | Confluent Tableflow (topics written straight to tables) | Hive tables |
| Push | WebSocket, SSE, Azure SignalR, Socket.IO | GraphQL subscriptions at scale | Long-polling |
| All-in-one platforms | Confluent Cloud, Databricks | Microsoft Fabric Real-Time Intelligence (generally available since 2024) | HDInsight |
The safe default
If you have to pick today
Debezium for change capture, Kafka or Event Hubs for the log, Flink only where you need joins or time windows, ClickHouse for serving, Iceberg or Delta for history, and SSE to the screen. Every piece is widely used in production, offered by more than one vendor, and easy to hire for. To place them, see the components and the eight designs.
FAQ
Questions people ask
What is the standard technology stack for real-time analytics?
Debezium for change capture, Kafka (or a cloud log such as Event Hubs or Kinesis), Flink or Spark for processing, ClickHouse, Pinot or Druid for serving, Iceberg or Delta for history, and WebSocket or SSE to the screen.
Is ksqlDB still a good choice?
Not for new builds. Confluent moved its stream processing to Flink, and ksqlDB is no longer where the investment goes.
Kafka or Redpanda?
Kafka is the established choice with the widest support. Redpanda speaks the Kafka protocol and is simpler to run; pick it with a reason, such as a small team or lower latency.
Next
Where to go next
↑ All of real-time analytics · Data platform architecture · Key concepts