Kafka as a Platform: the Ecosystem from the Ground Up
summary
Explains Kafka as a distributed immutable commit log that stores events as key/value bytes across topics, partitions and replicated brokers; shows how Producer and Consumer APIs append and read by offset, how partition keys guarantee ordering within a partition, and how consumer groups scale processing by partition assignment. Covers Kafka Connect for pluggable ETL, Schema Registry to enforce Avro/Protobuf/JSON schemas, Kafka Streams and ksqlDB for stateful stream processing, retention/compaction policies, and Confluent Cloud and tiered storage for long-term retention.
AI-written summary · about
chapters (22)
- 0:00Introduction
- 1:46Events
- 4:32Eventdriven architecture
- 9:55Topics
- 11:50Partitions
- 12:58Publish
- 16:38Read Data
- 24:16Kafka Brokers
- 26:45Kafka Ecosystem
- 29:09Kafka Connect
- 30:52Schemas
- 33:55Reading Data
- 39:07Kafka Events
- 41:27Questions
- 42:45Kafka Messages
- 43:30Ordering
- 44:41Dead Letter
- 45:17Kafka Partitions
- 46:23Kafka vs Messaging Systems
- 47:47Topic Limits
- 50:22Kafka Offsets
- 51:27Outro
related talks
41mKafka as a Platform: the Ecosystem from the Ground Up
41mKafka as a Platform: The Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
54mKafka: The Must Know Pub/Sub
46mBrick-by-Brick: Exploring the Elements of Apache Kafka®
01h 07mKafka Streams and KSQLDB
39mAn introduction to Apache Kafka - this is Event Streaming, not just Messaging
50mIntroduction to Apache Kafka