Building a Microservice Application Using Kafka and Django
summary
A practical walkthrough of using Apache Kafka as the data bus for a Django-based microservices system to handle high-throughput, fault-tolerant event streams. Covers integrating Kafka (with Avro and Confluent Schema Registry), Kafka Connectors/Debezium for PostgreSQL, Kafka Streams, and Python clients (confluent-kafka, python-kafka, pykafka, aiokafka), plus interoperability with services written in aiohttp, Go, etc., to tame scaling and resilience challenges.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 5:41What is Kafka
- 15:22Kafka architecture
- 20:23Microservice problems
- 24:42Kafka and Debezium solution
- 26:05Python Kafka libraries
- 28:42Producer code example
- 31:08Consumer code example
- 32:12Summary
- 47:51Q&A
related talks
59mBuilding Data Intensive Microservices with Apache Kafka
27mHow to Scale Your Services with Kafka
50mBuilding Data Streaming Platform with Apache Kafka
40mUsing Kafka in Your Python Applications
33mMicroservices, Events, and Breaking the Data Monolith with Kafka
26mEvent-driven applications: Apache Kafka and Python
42mCreating event-driven microservices: the why, how and what
41mApache Kafka + Apache Mesos = Highly Scalable Streaming Microservices