The Streaming Plane
summary
Maps the claim that a streaming plane actually delivers low‑latency realtime analytics by combining Apache Pinot style realtime OLAP with Kafka Redpanda Pulsar replication, shows materialized views that are continuously and incrementally updated, and demonstrates regional replication (EMEA → APAC) so local apps read without cross‑region latency. Gives concrete tools (connectors, stream processors, streaming databases), contrasts synchronous DB serving vs asynchronous Kafka subscriptions, and reports that push queries yield low latency while pull queries preserve flexibility.
AI-written summary · about
chapters (14)
- 0:00intro
- 0:14overview of key concepts
- 1:24understanding the data divide
- 2:52operational vs analytical data planes
- 3:49challenges in data movement
- 4:30introduction to data gravity
- 5:22the streaming plane explained
- 6:07synchronous vs asynchronous data serving
- 7:44data locality and replication
- 8:28materialized views and query optimization
- 9:16balancing push and pull queries
- 11:48global replication and local consumption
- 12:45streaming data catalog and governance
- 13:31conclusion and summary
related talks
34mReal-Time Analytics: Going Beyond Stream Processing with Apache Pinot
46mOptimizing Speed and Scale of User-Facing Analytics w Kafka and Pinot
39mFaster Analytics for Fast Data with Apache Pinot and Flink SQL
34mReal time OLAP and Stream Processing Friends or Foes
48mUnlocking Real-time Insights: Enhancing Your Databases With Stream Processing
49mIntroduction to Real-Time Analytics with Apache Pinot
47mReal-time analytics for your users with Apache Pinot
40mThe Seven Deadly Sins of Streaming