Connecting distributed data with Apache Kafka

on YouTube ↗26m

" summary

Shows Confluent Cloud on Azure moving on‑prem Kafka into a hybrid pipeline, using ksqlDB enrichment to join clickstream page views with a user DB and produce curated topics. Demonstrates the Cosmos DB sink connector provisioning, mapping the accomplished_female_readers topic to a Cosmos container, using a Kafka metadata id strategy (topic/partition/offset) for unique ids, and verifying records in the reader_analytics database. Mentions schema registry, multi‑language clients, provisioning cost estimate and a 99.95% SLA.

AI-written summary · about

" chapters (7)

1. 0:00Intro
2. 0:47Changing the world
3. 4:21Microsoft Partnership
4. 11:48Demo Description
5. 13:13Demo Setup
6. 15:08Topics
7. 17:53Data Integration

" related talks

• 14mPower real-time data streams from anywhere to Cosmos DB with ConfluentFlorence Sullivan · Microsoft Build 2023
• 41mApache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · All Things Open 2020
• 29mApache Kafka and ksqlDB in Action : Let’s Build a Streaming Data PipelineRobin Moffatt · DevOps Pro Europe 2021
• 31mBridge to the CloudSteve Howard · Indy Cloud Conf 2020
• 49mLet’s Build a Streaming Data Pipeline!Robin Moffatt · DataXDay 2020
• 43mApache Kafka and KSQL in Action: Let’s Build a Streaming Data Pipeline!Robin Moffatt · LISA18
• 40mA Data Mesh Of StreamsBen Stopford · State of Data Mesh 2022
• 53mApache Kafka and KSQL in Action : Let’s Build a Streaming Data Pipeline!Viktor Gamov · Jfokus 2019