Speeding Up the City with Apache Flink
summary
Apache Flink ingests Madrid open‑data streams (M30 highway + city sensors), keys events by station ID, unions streams and applies 5‑minute sliding windows checked every 2 minutes to emit per‑station alerts and area aggregates to Elasticsearch/Kibana. The pipeline uses Flink watermarks for late arrivals, supports exactly‑once delivery, allows early trigger firing on rising congestion, and the demo is implemented in ~100 lines of Scala running on a laptop VM with real‑time processing of dozens–hundreds of stations and claimed throughput ~100× Storm with low latency.
AI-written summary · about
chapters (7)
- 0:00Start
- 5:09BIG DATA MATURITY
- 9:08KEY-BASED WINDOWS
- 11:49TIME MATTERS
- 17:25PERFORMANCE
- 21:45Demo: Traffic in Madrid
- 28:50Emitting alerts
related talks
43mA Data Streaming Architecture with Apache Flink #bbuzz
45mApache Flink — Real-Time Data Processing
46mSmart Transit: Real-Time Transit Information with FLaNK
22mDo Flink on Web with FLOW
27mIntroduction to Flink in 30 minutes
38mApache Flink: A Very Quick Guide
49mscale.bythebay.io: Stephan Ewen, Apache Flink and the Next Wave of Stream Processing Applications
27mSolving Madrid's parking problem with Airflow and Streamlit