Event driven communication with Kafka on Kubernetes
summary
Explains running Apache Kafka on Kubernetes to implement an event-driven restaurant microservices system where order-created topic and delivery-assign topic carry persistent events that let orders queue while services are offline; uses Kafka Connect and MirrorMaker for DB and cluster integration (examples: MySQL/Postgres, S3/GCS, Elasticsearch, BigQuery/Snowflake), KRaft replaces ZooKeeper, and operators (e.g., Strimzi/KUDO-style) provision clusters with TLS/auth, autoscaling, and cert rotation. Claims LinkedIn-scale throughput (~100+ brokers, ~7 trillion messages/day) and shows that messages persist, replay, and enable resilient decoupled processing.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:55Kafka overview and features
- 9:23Event-driven architecture basics
- 11:53Why use Kafka on Kubernetes
- 17:42Restaurant microservices demo
- 26:23Kafka on Kubernetes challenges
- 28:11Summary and benefits
- 28:33Q&A
related talks
26mProduction Grade Kafka on K8s
42mCreating event-driven microservices: the why, how and what
41mMicroservices and Event Sourcing with Kafka and Kubernetes
33mMicroservices, Events, and Breaking the Data Monolith with Kafka
42mProcessing Millions of Events per Second with Apache Kafka on Kubernetes
29mKafka on Kubernetes: Does it really have to be the hard way?
21mHow Apache Kafka® Generates Value In Retail With Confluent
13mScaling Kafka on Kubernetes