The easy way to build and run Flink SQL jobs using Cloudera SQL Stream Builder

on YouTube ↗32m

" summary

Demo shows SQL Stream Builder running on Apache Flink inside Cloudera, using DDL to create tables from a faker connector and Kafka topics, auto-detecting schema and launching live Flink jobs that actually run. It builds materialized views backed by an internal Postgres-indexed engine with REST and PostgreSQL wire access for Grafana, supports integrated CDC via embedded Debezium without requiring Kafka, and lets users add Java and JavaScript UDFs. Jobs execute and materialized views serve pull queries with automatic indexing and pruning.

AI-written summary · about

" chapters (20)

1. 0:00Introduction
2. 0:18Demo overview
3. 1:02Why do we need SQL
4. 1:58What is SQL Stream Builder
5. 2:55Typical data flow
6. 4:27Complex data flow
7. 5:02materialized views
8. 6:13architecture
9. 7:48Integrated change data capture
10. 9:32Demo
11. 11:54Running the job
12. 13:05Creating a view
13. 15:28Kafka
14. 18:14Summary
15. 18:37Output
16. 19:42File systems
17. 23:46Create a materialized view
18. 25:02Ignore nulls
19. 26:42Connecting via Postgres
20. 28:34Advanced features

" related talks

• 39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · Data Con LA 2021
• 46mFlink SQL in 2020: Time to show off!Timo Walther, Fabian Hueske · Flink Forward Virtual 2020
• 43mContinuous SQL with Kafka and FlinkTimothy J Spann · JCON WORLD ONLINE 2023
• 23mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · Berlin Buzzwords 2021
• 23mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · DataEngBytes 2021
• 21mFlink's application at DidiXue Kang · Flink Forward Virtual 2020
• 43mLeverage the power and simplicity of SQL on Apache FlinkFabian Hueske · Berlin Buzzwords 2018
• 39mChangelog Stream Processing with Apache FlinkTimo Walther · Berlin Buzzwords 2022