"Kafka's Metamorphosis"

on YouTube ↗18m

" summary

SoftSwiss replaces RabbitMQ with Apache Kafka to handle hundreds of thousands of messages per second, proving it by wiring Postgres WAL into Kafka with Debezium so Rails needs no producer code. Production load hits ~500 requests/sec and player DBs exceed 1 TB; an unsupported inet type is patched upstream. The pipeline powers near‑real‑time finance reports, reduces development effort, and runs in production with no outages for three months.

AI-written summary · about

" chapters (9)

1. 0:00Introduction and background
2. 1:09Real-life scaling challenges
3. 3:40Monolith limitations and fixes
4. 5:27Why switch from RabbitMQ
5. 6:10Introducing Kafka
6. 8:44Testing Kafka on small scale
7. 10:54Results and benefits
8. 12:02Future plans and scaling
9. 12:49Q&A session

" related talks

• 38mKafka Message Broker Under High LoadArtem Vybornov · РИТ++ 2017
• 39m400 Billion Messages a Day with KafkaMAULIN VASAVADA, Kevin Lu, Na Yang · Silicon Valley Code Camp 2018
• 53m(Spring) Kafka - one more arsenal in a distributed toolboxNakul Mishra · Spring I/O 2018
• 01h 31mDeveloping Real-Time Data Pipelines with Apache KafkaJoe Stein · SpringOne2GX 2015
• 35mPolyglot, Fault-Tolerant Event-Driven Programming with Kafka....Natan Silnitsky · Devoxx Ukraine 2020
• 53mDemystifying KafkaNakul Mishra · JDD 2019
• 50mApache Kafka + Zookeeper = 3.5 Million Writes Per Second by DesignRanganathan "Ranga" Balashanmugam · Great Indian Developer Summit 2015
• 46mKafka 4, Fantastic?Damien Lucas · DevFest Toulouse 2025