Kafka-Backed Elixir at Scale
summary
Fresha runs Kafka for six years handling tens to hundreds of millions of messages per day across 110 services, and standardizes on kafka_ex / kox plus a small internal framework to avoid boilerplate. Domain guarantees use Postgres outbox plus Debezium WAL publishing and an Inbox table for idempotency. Protocol support is code-generated: 50k lines generated code from <1000-line generator, enabling a Kafka 4 jump implemented as 150 lines in 2–3 days. Tests use Testcontainers and mimic.
AI-written summary · about
chapters (7)
- 0:00Kafka and Elixir intro
- 5:01Developer experience gains
- 9:59Resilience in event-driven
- 15:29Elixir and OTP for Kafka
- 21:07Kafka binary protocol
- 35:20Key takeaways and goals
- 37:29Q&A on testing
related talks
41mKafka as a Platform: The Ecosystem from the Ground Up
35mKafka Deployment at Scale
01h 00mKafka as a Platform: the Ecosystem from the Ground Up
39mKafka as a Platform: The Ecosystem from the Ground Up
01h 00mKafka as a Platform: the Ecosystem from the Ground Up
35mSo You’ve Got Yourself a Kafka: Event-Powered Rails Services
53mDemystifying Kafka
29mKafka on Kubernetes: Does it really have to be the hard way?