Shifting Safety Techniques To a Statistical World
summary
Argues that conventional deterministic assurance breaks on high‑performance, non‑deterministic systems and replaces it with statistical claims backed by data: uses F3 traces and hashing to measure path frequencies, fits a power‑law to rare traces and applies Simple Good‑Turing to estimate probability of untested paths, computes a conservative beta factor for common‑cause failures, leverages SHAP for ML bug classifiers and line‑to‑shas/patch‑impact tester for config filtering, and reports that a ~320‑day kernel removes ~75% of known bugs and yields ~70 residual bugs per sublevel (~1 bug / 3k LOC).
AI-written summary · about
chapters (13)
- 0:00Statistical safety techniques intro
- 2:50Traditional safety challenges
- 5:19Statistical path coverage
- 9:57Nondeterminism for redundancy
- 11:41Open-source process assessment
- 13:04Linux kernel bug lifetime
- 17:45Using machine learning to classify software bugs
- 21:17ML model assurance
- 24:03Kernel selection with stats
- 25:51Residual bug estimation models
- 29:50Replacing safety measures
- 33:28Justifying stats in standards
- 36:43Q&A
related talks
46mHow Cheap Learning (not deep learning) Finds Faults in Distributed Systems
39mTrusting Your Data Science Pipelines: A Quantitative Approach
24mFuzzing Guided by Bayesian Program Analysis
38mApplying machine learning to software development to reduce bugs
48mUsing Machine Learning to find Linux bugs
25mAI Reliability in Security: A New Data Science Paradigm
38mHolistic root cause analysis of software breakages through structural causal modeling
31mFinding the Needle in the Haystack: Predicting Storage Device Failures in...