Lessons learned from working with 2000 event-driven microservices

on YouTube ↗54m

" summary

Wix runs ~70 billion Kafka events per day across ~2,500 microservices and reports concrete fixes: a homegrown resilient producer (fallback to S3 + Healer) and the Greyhound client, default compression and S3-for-large-payloads, and schema fields like revision ID and transaction ID to prevent out‑of‑order and duplicate effects. Built production tools to search events, inspect partition lag, and skip or replay messages; claims near-zero producer loss and measurable reductions in consistency incidents.

AI-written summary · about

" chapters (9)

1. 0:00Introduction to event-driven
2. 4:18Event-driven architecture basics
3. 10:15Solving message production issues
4. 14:22Handling out-of-order events
5. 19:26Ensuring idempotency
6. 22:27Managing large payloads
7. 26:12Observability and tooling
8. 31:58Summary and future outlook
9. 34:50Q&A session

" related talks

• 46mLessons Learned From Working with 2000 Event-Driven MicroservicesNatan Silnitsky · Build Stuff 2022
• 30m8 Lessons Learned from using Kafka with 1000 microservicesNatan Silnitsky · Scale By the Bay 2020
• 43mBattle-tested event-driven patterns for your Microservices archit.Natan Silnitsky · Devoxx Ukraine 2020
• 41mBattle-tested event-driven patterns for your microservicesNatan Silnitsky · Scale By the Bay 2021
• 35mPolyglot, Fault-Tolerant Event-Driven Programming with Kafka....Natan Silnitsky · Devoxx Ukraine 2020
• 52m10 LESSONS LEARNED FROM USING KAFKA IN 1000 SCALA MICROSERVICESNatan Silnitsky · Scalar 2020
• 37mKafka Based Global Data Mesh at WixNatan Silnitsky · DevDays Europe 2022
• 44mScaling Horizons Effective Strategies for Wix's Scaling challengesNatan Silnitsky · GeeCON 2024