A Year in Flink - Flink SQL Live Coding
summary
A live Flink SQL demo rebuilds community stats by ingesting GitHub commits, PRs and three mailing‑list mbox archives into Kafka then querying them from the Verica platform editor and a session Flink cluster. Queries use tumbling 30‑day windows and distinct counts to produce monthly metrics: distinct mailing‑list users, total emails, and threads (average ~4.0 emails per thread). The Chinese user list shows higher volume; the dev list query runs as a retract stream and writes top committers into Elasticsearch (top anonymous/hidden email 160 commits, next 32 and 16). Flink SQL live coding GitHub API -> Kafka -> Flink
AI-written summary · about
chapters (13)
- 0:00Welcome and introduction
- 1:53Why automate statistics
- 2:34What is Flink SQL
- 6:27Setting up the demo
- 7:51Analyzing mailing lists
- 12:56Counting emails per month
- 13:59Measuring email thread engagement
- 20:06Comparing mailing lists
- 21:31Joining all mailing lists
- 26:32Live SQL job execution
- 27:34Deep dive into dev mailing list
- 28:39Most active committers
- 30:03Q&A and summary
related talks
46mFlink SQL in 2020: Time to show off!
37mA Year in Flink: The Most Important Changes in Recent Versions
25mOnly SQL: Empower data analysts end-to-end with Flink SQL
39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
43mContinuous SQL with Kafka and Flink
23mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
27mIntroduction to Flink in 30 minutes
43mLeverage the power and simplicity of SQL on Apache Flink