Interactive Session: Running Flink SQL jobs in production
summary
Demo shows a Flink-based product wiring Kafka to external stores (Kudu, Hive) with an auto-detect Kafka table wizard and a Faker connector to generate data, then performs a temporal lookup join (latest-version enrichment) with optional caching. Covers operational details: savepoint compatibility limits for evolving SQL, metrics/logs shipped via Kafka into ELK/Prometheus, auth via Knox/Ranger, and exposing results as materialized views via Postgres wire protocol for third-party dashboards and alerting.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:25Creating an order table
- 5:50General discussion
- 9:45Flink Wizard
- 13:40Join strategies
- 15:33Other components
- 18:34materialized views
- 23:16interoperability
- 26:06SQL jobs
- 29:16Logs metrics
- 32:56Questions
related talks
46mFlink SQL in 2020: Time to show off!
43mContinuous SQL with Kafka and Flink
43mLeverage the power and simplicity of SQL on Apache Flink
39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQL
19mSupercharging Kafka Applications with Real-time Contextual Insights
23mFrom Streams to Tables and Back Again: A Demo of Flink’s Table & SQL API
31mEnd-to-end Flink SQL with Schema Registry, Kafka & Kudu
41mBeyond the Hype: When Apache Flink Solves Real Problems