Getting Started with Event Stream Processing via Apache Flink on Microsoft Azure

on YouTube ↗20m

" summary

Apache Flink runs batch and streaming from the same APIs and deploys on Azure via VM Scale Sets, Azure Kubernetes Service or partner platforms (Cloudera, Verica); there is currently no fully managed Flink on Azure. Integrations include Kafka endpoints (Confluent Cloud, Event Hubs, HDInsight), sinks via JDBC to Postgres/MySQL/SQL Server, Cosmos DB (Mongo/Cassandra APIs) and streaming file sink to Blob/ADLS Gen2 targeted for Flink 1.14. Use cases shown: retail inventory, finance claims, healthcare and IoT for near‑real‑time processing.

AI-written summary · about

" chapters (6)

1. 0:00Introduction and welcome
2. 1:54Motivations for stream processing
3. 6:07How stream processing works
4. 10:22Deployment options on Azure
5. 12:31Integration with Azure services
6. 14:40Summary and getting started

" related talks

• 44mApache Spark on Azure - solution (...)Kamil Dworak · 4Developers Katowice 2019
• 34mBeaming Flink to the Cloud @ NetflixMonal Daxini · Flink Forward Berlin 2016
• 48mStreaming Azure ServicesPedro Mendes, Miguel Jesus · Global Azure Portugal 2021
• 26mRunning Apache Flink on AWSRyan Nienhuis · Flink Forward Europe 2019
• 44mBuild a Real-time Stream Processing Pipeline with Apache Flink on AWSSteffen Hausmann · Flink Forward Berlin 2017
• 40mFrom concept to production; the fastest way to develop & deploy Apache Flink jobsSteffen Hausmann, Danny Cranmer · Flink Forward Global 2021
• 30mBuild and run streaming applications with Apache Flink and Amazon Kinesis Data AnalyticsHausmann · Flink Forward Europe 2019
• 51mImplementing an Event Sourcing strategy on AzureOlena Borzenko, Eldert Grootenboer · Build Stuff 2021