Continuous Queries on Apache Pulsar using Flink SQL

on YouTube ↗25m

" summary

StreamNative Cloud provisions an Apache Pulsar cluster and a Flink (Blink) SQL cluster, wires them together, and runs live SQL: a created topic feeds a SELECT and a grouped streaming aggregation that updates in real time; the demo boots clusters from the console, shows results streaming to the UI, then tears down the demo cluster. Integration uses the Pulsar SQL/Flink connector with exactly-once semantics, supports tiered storage offload to S3/GCS and Kafka protocol compatibility, and the Flink 1.14 source merged* is in progress.

AI-written summary · about

" chapters (3)

1. 0:00Intro
2. 0:54Apache Pulsar & StreamNative
3. 6:49Flink SQL on StreamNative Cloud

" related talks

• 47mQuery Pulsar Streams using Apache FlinkSijie Guo · Flink Forward Europe 2019
• 39mCodeless Pipelines With Pulsar and FlinkTimothy J Spann · Data Con LA 2021
• 41mDeep Dive into Building Streaming Applications with Apache PulsarTimothy J Spann · CloudOpen @ Open Source Summit Europe 2022
• 39mBetter, Faster, Stronger Streaming: Your First Dive into Flink SQLCaito Scherr · Data Con LA 2021
• 14mFast Data on-Ramp with Apache Pulsar on K8Timothy J Spann · Kubernetes Batch + HPC Day Europe 2022
• 28mBuild your next-generation stream platform based on Apache PulsarNeng Lu · Flink Forward Virtual 2020
• 48mApache Pulsar: Event Streaming in The Cloud Native WorldMary Grygleski · GOTO Chicago 2023
• 46mFlink SQL in 2020: Time to show off!Timo Walther, Fabian Hueske · Flink Forward Virtual 2020