How Not to DDOS Servers
summary
A Kafka outage causes a half-hour backlog to replay into a new Go HTTP-delivery service, producing about 4× normal request rate and driving downstream apps and the shared load balancer to fail; the sensible remedy is concrete: put rate limiters between Kafka and the delivery service and between the delivery service and apps, cap bursts (they use 250 requests per second as a working upper bound), and add circuit breaker behavior; a reusable Go library for these shunts exists and is offered.
AI-written summary · about
chapters (7)
related talks
04mPREVIEW: Running Kafka for Maximum Pain
02h 48mKafka: Map of Pitfalls for Developers and Ops
28mTackling Kafka, with a Small Team
43mAttacking (and Defending) Apache Kafka
42mRunning at 99% Surviging an Application DoS
34mHow to Survive to High Demanding Services
26mBeing good neighbors - Rate limiting in a serverless world
30mThe last in the line: a talk about DevOps and Disasters