Interactive Session: Beam and Beam on Flink general roadmap discussion and AMA
summary
Interactive AMA covers the Beam-on-Flink roadmap with concrete claims: cross-language portability now lets Java connectors be used from Python and Go, the Go SDK usable is promoted as production-ready, TensorFlow Extended runs heavy Beam pipelines so TFX uses Beam in real deployments, and deployment patterns favor container images and community Flink operators rather than a Beam operator—Kubernetes operator examples are linked. Work continues on pushing filters early and on tighter integration with Flink’s Table API/Calcite to let the runner pick join algorithms and yield large performance wins.
AI-written summary · about
chapters (27)
- 0:00Intro
- 0:30Ken
- 2:19Beam overview
- 4:14Beam on Flink
- 5:28Beam on Flink presentations
- 7:02Python support
- 8:56Kotlin
- 9:15Joshua
- 11:39kajillion
- 12:03Beam philosophy
- 12:39Beam runners
- 13:48Deployment
- 15:47Thirdparty libraries
- 16:35Customer perspective
- 17:12QA
- 17:31Upcoming features
- 19:25Summary
- 21:22Integration with Flink
- 22:23Joint algorithms
- 24:09Optimizing relational operations
- 24:47Beam architecture
- 29:33Development workflows
- 29:53Getting started
- 30:25SQL standardization
- 30:44Formal vs informal standardization
- 36:46Calcite
- 37:07Tablevalued functions
related talks
33mFlink and Beam: Current State & Roadmap
21mApache Beam (incubating)
31mMaking Big Data Processing Portable. The Story of Apache Beam and gRPC - Ismaël Mejía
47mApache Beam - what do I gain?
41mPython, Java, or Go: It's Your Choice ...
41mApache Beam: Unified Data Processing
29mApache Beam Making Big Data Portable With gRPC
40mIt’s finally here: Python on Flink & Flink on Zeppelin