Event-Driven applications: Apache Kafka and Python
summary
Practical walkthrough of Apache Kafka with Python showing actual code: use kafka-python producer/consumer code to write JSON/Avro bytes, create topics with partitions, observe ordering per partition, and set retention and replication (replication factor of three shown). Demonstrates scaling by partitioning, consumer groups for non-overlapping reads, and a live demo using Kafka Connect JDBC sync to Postgres that creates the target table and immediately populates three rows then a fourth — connector creation and replication work as shown.
AI-written summary · about
chapters (20)
- 0:00Intro
- 3:10Event-Driven Applications
- 3:39What is an Event?
- 6:13What is Kafka?
- 7:49Distributed
- 9:07What is a Kafka Event?
- 10:55Write to Kafka
- 11:32Read from Kafka
- 18:32Pizza Demo!
- 19:04Topic Retention Policies
- 19:50Log Size
- 20:52Topic Partitions
- 22:23Selecting a Partition
- 23:05Ordering
- 25:22Scaling Out
- 26:31Partitions Demo!
- 30:16Multiple Applications
- 33:29Evolve Existing Applications
- 34:10Distribute Events
- 34:58Kafka Connect Demo!
related talks
38mEvent-driven applications: Apache Kafka and Python
26mEvent-driven applications: Apache Kafka and Python
27mReal-time Event Streaming with Python
40mUsing Kafka in Your Python Applications
36mKafka Streaming, the Pythonic Way
30mDiving into Event-Driven Architectures (EDA) with Python - Solving complexity at scale
51mSimplifying Event-Driven Services with FastStream: A How-To Guide
26mBuilding A Python Service Using Open Source Components That Connect, Transform & Deliver Data In RT