Fault Tree Analysis Applied to Apache Kafka

on YouTube ↗31m

" summary

Applies fault tree analysis to Kafka clusters and gives concrete numbers: assumes 4% annual disk failure, 2% network, 1% hardware and OS faults, computes a single-broker failure ~12.8%, shows three brokers + three Zookeepers on SSDs reaches four nines availability, cites AWS EBS at 0.2% as an alternative, models remediation (three-day and longer) and replica rebuild rates (70 MB/s ⇒ 24 hours for 6 TB, 8 days for very large brokers) and shows when durability drops below 7 nines; diagrams and scripts on GitHub (scram / FTA-Kafka).

AI-written summary · about

" chapters (6)

1. 0:00Introduction to Kafka SLOs
2. 3:21Why Kafka reliability matters
3. 8:22Fault tree analysis basics
4. 17:56Analyzing Kafka availability
5. 23:23Ensuring data durability
6. 27:35Latency and final thoughts

" related talks

• 13mScaling Kafka on KubernetesShalini Katyayani Koney · Conf42 Kube Native 2025
• 53mDemystifying KafkaNakul Mishra · JDD 2019
• 41mUnderstanding the CAP Theorem with Apache KafkaSébastien Guilloux · DevFest Toulouse 2018
• 01h 12mRedundancy does not imply fault toleranceAishwarya Ganesan · Hydra 2020
• 38mRe-architecting Apache Kafka for the CloudGwen Shapira · Craft Conference 2021
• 24mMake Your Kafka Cluster Production-ReadyJakub Scholz · Data on Kubernetes Day North America 2023
• 41mKafka as a Platform: The Ecosystem from the Ground UpRobin Moffatt · GOTOpia November 2020
• 47mRunning Kafka in Kubernetes: a practical guideKatherine Stanley · Devoxx Belgium 2019