Reliable Toolkit for Event-Driven Microservice Architecture

on YouTube ↗45m

" summary

Describes splitting a monolith into event-driven microservices using Kafka on AWS MSK and later a self‑managed Kafka cluster, with concrete tooling: CDC via Kafka connectors and a Transactional Outbox pattern to prevent lost events. Messages are JSON with versioned schemas loaded from a schema registry at startup; observed publish latency is typically 5–10 ms with 99th-percentile spikes near 1 s. A Smart Dead Letter Queue holds failed messages for a partition key and uses counters to stall subsequent messages until fixes are replayed in order.

AI-written summary · about

" chapters (14)

1. 0:00Toolkit intro
2. 5:04Critical drivers
3. 6:56Monolith migration
4. 10:06Microservice choice
5. 12:36Sync vs async
6. 17:07Async benefits
7. 20:22Queue vs streaming
8. 22:52Kafka risks
9. 25:28Schema versioning
10. 29:25Kafka fault tolerance
11. 32:39Outbox publishing
12. 34:43Change Data Capture
13. 37:36Message ordering
14. 41:30Q&A

" related talks

• 42mCreating event-driven microservices: the why, how and whatKatherine Stanley · Voxxed Days Microservices 2019
• 33mMicroservices, Events, and Breaking the Data Monolith with KafkaBen Stopford, Cornelia Davis · Cloud Foundry Summit Europe 2019
• 54mMicroservices with KafkaAlbert Laszlo Robert · Voxxed Days Singapore 2017
• 11mLightning Talk: Event Based Communication in Complex Micro-service...Sapna Upreti, Prabal Raghav · OpenJS World 2021
• 31mEvent driven communication with Kafka on KubernetesRaihan Khan · KCD Dhaka 2024
• 21mHow Apache Kafka® Generates Value In Retail With ConfluentAnna Semjen · Big Data LDN 2022
• 01h 08mEvent Driven Architecture – Enabling MicroservicesBrad Irby · Silicon Valley Code Camp 2018
• 47mEvent-Driven Microservices = Microservices Done RightUrs Peter · J-Fall 2022