A Data Streaming Architecture with Apache Flink #bbuzz

on YouTube ↗43m

" summary

Shows converting a batch ETL pipeline into a streaming ETL built with Apache Flink reading from Apache Kafka and writing to Elasticsearch, Cassandra and a per-minute rolling file sink. Demonstrates processing 500 MB of historical tweets with event-time watermarks, 10s tumbling windows for language counts and a one‑minute sliding window for top‑N word frequencies, using Flink state backends, checkpoints and an exactly-once Kafka connector so windows and replays produce correct results despite out‑of‑order and late arrivals.

AI-written summary · about

" chapters (7)

1. 0:00Introduction to Flink
2. 2:57Why streaming is transformative
3. 11:43Flink features overview
4. 12:06Transforming batch ETL to streaming
5. 20:14Streaming ETL demo setup
6. 23:29Running the streaming ETL job
7. 31:26Closing and resources

" related talks

• 38mApache Flink: A Very Quick GuideVladimir Bacvanski · Data By the Bay 2016
• 42mApache Flink deep-dive #bbuzzStephan Ewen · Berlin Buzzwords 2015
• 49mscale.bythebay.io: Stephan Ewen, Apache Flink and the Next Wave of Stream Processing ApplicationsStephan Ewen · Scale By the Bay 2017
• 41mBeyond the Hype: When Apache Flink Solves Real ProblemsNaci Simsek · Berlin Buzzwords 2026
• 30mBuild and run streaming applications with Apache Flink and Amazon Kinesis Data AnalyticsHausmann · Flink Forward Europe 2019
• 50mStream Processing Smackdown: Kafka Streams vs. Apache FlinkViktor Gamov · Jfokus 2025
• 40mFrom concept to production; the fastest way to develop & deploy Apache Flink jobsSteffen Hausmann, Danny Cranmer · Flink Forward Global 2021
• 34mBeaming Flink to the Cloud @ NetflixMonal Daxini · Flink Forward Berlin 2016