Apache beam and google cloud dataflow

on YouTube ↗24m

" summary

Overview of Apache Beam and Google Cloud Dataflow, explaining how to build and run data-processing pipelines using Beam’s unified programming model and execute them on Google Cloud Dataflow as a runner for scalable, managed processing.

AI-written summary · about

" chapters (6)

1. 0:00Start
2. 3:22What is apache beam Unified APs that encapsulates the entire capabilities of data processing regardless the runner that preform the computation
3. 4:27Why use apache beam? Apache beam is very simple and intuitive It allows you to replace the data processing from one to another It allows you to move from on premise to the cloud and vice
4. 12:58Apache beam programming model Apache beam programming model includes: pipeline Pcollection, transformers and source & Sync
5. 13:28Pipeline When you write a program in apache beam you write pipeline.
6. 15:31Transformers Transformers are the business logic of the processing we have root transformers, core transformers and composite transformers

" related talks

• 39mApache Beam and Google Cloud DataFlowMete Atamel · GDG DevFest Ukraine 2017
• 26mBeam and dataflow: a perfect combination of parallel processingEl Mehdi El Khayati · BlaBlaConf 2022
• 52mApache Beam, a unified model for Big Data processingJean-Baptiste Onofré, Ismaël Mejía · DevFest Nantes 2016
• 10mDemocratizing the pipeline with Apache Beam and DataflowTyler Akidau, Reuven Lax · Strata Data Conference 2017 - New York, New York
• 47mApache Beam - what do I gain?Łukasz Gajowy · GeeCON 2019
• 41mApache Beam: Unified Data ProcessingAlexey Diomin · Voxxed Days Minsk 2018
• 53mStreaming data processing pipelines with Apache Beam [in Python, naturally!]Markku Lepisto · PyCon APAC 2018
• 46mBuilding an Apache Beam streaming runtime in GCPSergei Sokolenko, Reuven Lax · Flink Forward Europe 2019