HDDse: Enabling High-Dimensional Disk State Embedding for Generic Failure Detection
summary
HDDse uses an LSTM-based siamese network to map daily SMART snapshots into unified high-dimensional disk state embeddings and classifies similarity pairs with a voting sliding window over time. It generates huge training pairs (combining snapshots reduces class imbalance and helps minority models with <1,500 disks), adapts to unseen disk models without per-model retraining, and outperforms prior methods on Tencent and Backblaze data, improving a cost-aware MTTDL metric by about two orders of magnitude while keeping practical training and detection time.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 0:38Device failure
- 1:57Six Classes
- 2:24Eliminations
- 4:37Preliminary Results
- 9:36Overview
- 10:22Structure
- 10:50Sample Pool
- 12:29Improved Robustness
- 13:51Experimental Results
- 14:52HDDse Performance
- 17:29Field Time Statistics
- 18:11LongTerm Availability
- 18:51Conclusion
related talks
31mFinding the Needle in the Haystack: Predicting Storage Device Failures in...
21mExploit both SMART Attributes and NAND Flash Wear Characteristics to Effectively...
30mCluster storage systems gotta have HeART...
21mDataEngBytes 2023 - SYD-T2-04 - Suneeta Mall
29mDataEngBytes 2023 - SYD-T2-01 - Rene Essomba
46mML Use Cases at Think Big Analytics
22mHypeReca: Distributed Heterogeneous In-Memory Embedding Database for Training...
33mTowards a deeper understanding of retrieval and vector databases