StreamApprox: Approximate Stream Analytics in Apache Flink
summary
Prototype implements online stratified reservoir sampling for Apache Flink (also ported to Spark) and answers linear queries like sum and mean over streams while producing rigorous error bounds. Users submit an accuracy or latency budget and the system adapts sampling fraction with a feedback loop, computing error from the observed variance and sample size. Implementation requires no cross-worker synchronization, runs inline as a sampling operator, and evaluation reports substantial throughput gains versus Spark’s sampling approaches.
AI-written summary · about
chapters (7)
- 0:00Introduction to StreamApprox
- 7:50Why approximate computing
- 10:01System design overview
- 18:16Online sampling algorithm
- 20:28Evaluation results
- 23:47System properties and conclusion
- 25:03Q&A session
related talks
38mApache Flink: A Very Quick Guide
40mRobust Stream Processing with Apache Flink
28mSweet Streams Are Made Of These
42mDistributed Real-Time Stream Processing: Why and How
49mscale.bythebay.io: Stephan Ewen, Apache Flink and the Next Wave of Stream Processing Applications
39mEvaluating Streaming Framework Performance for a Large-Scale...
50mStream Processing Smackdown: Kafka Streams vs. Apache Flink
42mStream Processing beyond Streaming Data