I²: Interactive Real-Time Visualization for Streaming Data with Flink & Zeppelin
summary
I² connects Apache Flink and Zeppelin to drive live visualizations by sending UI changes back into the pipeline and adapting operators at runtime, using a coFlatMap control stream to merge control messages with data. It implements adaptive filters and can swap operator logic or thresholds without restarting jobs, and it uses a four-point per-pixel aggregation (first, last, min, max) to losslessly reduce 12k events/sec to a browser-friendly stream. Open source demo replays football sensor data.
AI-written summary · about
chapters (11)
- 0:00Introduction to I²
- 1:23Interactivity in I²
- 2:51Architecture overview
- 3:54Football match use case
- 4:38Live visualization demo
- 6:26Zeppelin integration
- 8:14Feedback stream mechanism
- 9:39Adaptive Flink operators
- 11:06Data reduction technique
- 14:20Performance comparison
- 15:44Summary and Q&A
related talks
44mApache Zeppelin: A friendlier way to Flink
34mAdvanced visualization of Flink and Spark jobs
39mHow We're Democratizing Stream Processing at Zillow
27mStream, Store, Visualize: Fast-Track Real-Time Analytics the OSS Way
43mHail Hydrate! From Stream to Lake Using Open Source
41mApache Zeppelin, the missing GUI for your Big Data backend
38mInteractive Session: Flink on Autopilot with Splunk
40mInteractive Session - Live Code Contribution to Apache Flink