Finding the Needle in the Haystack: Predicting Storage Device Failures in...
summary
Claims a practical, multi-phase active sampler plus learning pipeline that cuts raw telemetry by sampling high‑risk devices and retrains models on demand. Implementation uses Instana, Thermodynamic, EIM, smart stats and a benchmark model builder with an adaptive scheduler; tested on flash, HDD and tape in IBM China system center (Triton) lab. Uses Blackblaze historical baselines (~230,000 drives) and targets exabyte growth (175 zettabytes by 2025); POC produces alerts into buckets missing/read-only/slow and reports successful detections.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 3:10Storage failure prediction importance
- 5:17Storage media diversity challenges
- 8:07Storage media health factors
- 10:57Metrics and monitoring approaches
- 13:29Failure prediction strategies
- 16:44Sampling and anomaly detection
- 18:10Model components for prediction
- 20:34Model builder and scheduler
- 22:40IBM China System Center implementation
- 24:48Full stack observability
- 25:51Monitoring tools and analysis
- 27:17Predictive maintenance
- 29:26Sustainability insights
- 29:46Key takeaways and patent overview
related talks
14mFailureMiner: A Joint Key Decision Mining Scheme for Practical SSD Failure Prediction...
53mA SMART-er Ceph: Predicting Hard Drive Failure
21mExploit both SMART Attributes and NAND Flash Wear Characteristics to Effectively...
20mHDDse: Enabling High-Dimensional Disk State Embedding for Generic Failure Detection
12mRevolutionizing Data Management
27mMaking Disk Failure Predictions SMARTer!
46mShorten the Path to Production with In-Database Machine Learning
22mEngineering Predictive Platforms