Efficient Exposure of Partial Failure Bugs in Distributed Systems with Inferred...
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
Legolas statically instruments code with fine-grained injection hooks and automatically infers abstract state variables (ASVs) to compress millions of fault points into meaningful groups, then uses a budgeted round‑robin + randomized decision policy to pick injection times. Applied to six popular distributed systems, it finds 20 new partial‑failure bugs that elude coarse fault tests; the BSR policy exposes all 20 with median 58 minutes versus a new‑state‑only policy that finds 8 (median 11 minutes). The framework is implemented and open source.
AI-written summary · about
chapters (11)
- 0:00Legas framework overview
- 2:45Fault injection challenges
- 4:32Legas workflow
- 5:35Static analysis for hooks
- 7:26Abstract State Variables
- 9:53Extracting Abstract State Variables
- 11:19State-aware injection policies
- 13:10Legas evaluation on systems
- 14:39New bugs and confirmations
- 15:41Comparison with alternatives
- 16:50Conclusion and open-source
related talks
19mCAFault: Enhance Fault Injection Technique in Practical Distributed Systems via...
17mOne-Size-Fits-None: Understanding and Enhancing Slow-Fault Tolerance in Modern...
18mPreview: Verification (Correctness & Debugging)
40mCatching More Bugs with Fewer False Alarms
21mDependency Safety for Distributed Systems
04mOrchestrating chaos: Applying database research in the wild
14mUnderstanding and Detecting Fail-Slow Hardware Failure Bugs in Cloud Systems
14mFeedback-guided Adaptive Testing of Distributed Systems Designs