Spinning up micro-services using Ruby/Kafka

on YouTube ↗11m

" summary

Describes moving a large Rails monolith to an event driven design using Apache Kafka and a centralized publisher concern so DB changes become producers instead of scattered callbacks, with Rails concern for producers enforcing metadata and serializers. Uses Karateka + Sidekiq consumer pipeline to register controllers per topic and process messages. Implements a working signup flow publishing to a user_created topic and welcome SMS and demos it locally, claiming development is simple though deployments add operational work.

AI-written summary · about

" chapters (6)

1. 0:00The challenges of monoliths
2. 1:20Decoupling with events
3. 3:38Introducing Apache Kafka
4. 5:20Implementing producers
5. 7:52Processing with consumers
6. 8:58Live demo and conclusion

" related talks

• 11mSpinning up micro-services using Ruby/KafkaAnkita Gupta · RedDotRubyConf 2017
• 31mEvent Streaming on RailsBrad Urani · RailsConf 2022
• 35mApplying microservices patterns to a modular monolithGuillermo Aguirre · RailsConf 2023
• 27mScaling a Rails Monolith w/Event Driven MicroservicesMarko Anastasov · WindyCityRails 2016
• 38mEffective Data Synchronization between Rails MicroservicesAustin Story · RailsConf 2021
• 27mEvent Streaming Patterns for Ruby ServicesBrad Urani · RubyConf TH 2023
• 42mCreating event-driven microservices: the why, how and whatKatherine Stanley · Voxxed Days Microservices 2019
• 31mEvent driven communication with Kafka on KubernetesRaihan Khan · KCD Dhaka 2024