Let’s Build a Streaming Data Pipeline!
summary
Walks through a working streaming pipeline built with Apache Kafka and ksqlDB, showing Kafka Connect for CDC from MySQL, Avro + Schema Registry for serialization, and a sink to Elasticsearch/Kibana. Live demo enriches ratings events with customer state, filters for poor platinum reviews into a new topic, and drives notifications and a dashboard. All pieces are wired end to end and shown running, with topic creation, offsets, and metrics visible in the demo.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 2:17Kafka core concepts
- 9:11Why streaming pipelines
- 12:59Integrating with Kafka Connect
- 15:45Stream processing fundamentals
- 24:41Practical demo and walk-through
- 44:54Resources and closing
related talks
29mApache Kafka and ksqlDB in Action : Let’s Build a Streaming Data Pipeline
41mApache Kafka and ksqlDB in Action: Let’s Build a Streaming Data Pipeline!
26mConnecting distributed data with Apache Kafka
18mBuild a data pipeline in 20 minutes with Kafka Connect
37mBuilding an Event Analytics Pipeline with Kafka, ksqlDB, and Druid
57mTaming IoT Data: Making Sense of Sensors with SQL Streaming
39mKafka as a Platform: The Ecosystem from the Ground Up
08mPREVIEW: Kafka Stream Processing for Everyone with KSQL