Testing Level Hard: Distributed Apps
summary
Demo shows fault injection with blockade against a containerized app and reproduces a connection leak that triggers Redis "max clients" errors, caused by slow Nginx responses holding sockets. A reproduction of the scenario loses 10 lost tasks in 10,000 and exposes reconnection and single‑point‑of‑failure faults. The talk also points out that etcd dedicates over half its code to tests and that tc and iptables fault injection finds surprising, production‑visible bugs and operational limits worth fixing before deployment.
AI-written summary · about
chapters (4)
related talks
28mAdvanced Testing Techniques
41mData Driven and Containerized Testing of Multi Service Networked Systems
25mResilience testing in modernization projects
27mAutomating Resilience Testing with Clojure and Docker
28mBuilding and testing resilient applications
43mTaking testing to the next level
19mCAFault: Enhance Fault Injection Technique in Practical Distributed Systems via...
33mBug Hunting in Distributed Systems: Using Robustness Tests to Test...