Building a Real-time Streaming Platform Using Kafka Streams...
summary
Argues that all data can be modeled as table–stream duality and that Apache Kafka is a durable commit-log acting as a central streaming backbone. Describes Kafka Connect as a scalable connector framework that moves data in/out and preserves schemas (useful with Confluent’s schema registry) and describes Kafka Streams as an embedded Java library (requires Kafka 0.10 timestamps) that does low-latency stateful joins, aggregations and materialized views without a separate cluster, and shows how these pieces compose into a working streaming platform.
AI-written summary · about
chapters (9)
- 0:00Introduction to Kafka
- 1:20Understanding event streams
- 4:47Streaming platform benefits
- 8:03Core Kafka abstractions
- 11:34Apache Kafka Connect
- 15:30Kafka Streams paradigm
- 21:12Kafka Streams library
- 25:50Tables and data processing
- 28:46Ecosystem and Q&A
related talks
55mDemystifying Stream Processing with Apache Kafka
53mBuilding Event Driven Services with Apache Kafka and Kafka Streams
33mKafka Microservices with Kafka plus Akka and Kafka Streams
41mApache Kafka simply explained
40mDistributed Streaming Platforms in Modern Architectures with Apache Kafka
50mIntroduction to Apache Kafka
30mEvent Streaming and the Future of Applications
47mOne Does Not Simply Query a Stream