Boosting Full-Node Repair in Erasure-Coded Storage
summary
RepairBoost is a scheduling framework that speeds full-node repair for linear erasure codes by building a DAG repair abstraction, balancing upload and download across nodes with a tuple-based mapping, and casting chunk dispatch as a max‑flow transmission scheduling problem. Prototype implemented with cpuff path and integrated into HDFS runs on Amazon EC2 (1 coordinator + 16 agents, 1 Gbps, k=6, m=3) and shows 35.0–97.1% speedups (60.4% avg), with breakdown gains of 45.7% and 19.8% for balancing and scheduling, and small degradation for 1–3 concurrent failures.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:16Erasure Coding
- 2:43Repair-Efficient Codes
- 3:24Repair Algorithms
- 4:52Motivation
- 6:38Our Contributions
- 7:17Repair Abstraction
- 9:07Repair Traffic Balancing
- 10:54Transmission Scheduling
- 12:27Implementation
- 13:02Evaluation Setup
- 13:54Performance Results
- 14:23Breakdown Analysis
- 14:44Multi-Node Repair
- 15:07Conclusion
related talks
17mParaRC: Embracing Sub-Packetization for Repair Parallelization in MSR-Coded Storage
11mExploiting Combined Locality for Wide-Stripe Erasure Coding in Distributed Storage
18mDRBoost: Boosting Degraded Read Performance in MSR-Coded Storage Clusters
15mSelectiveEC: Selective Reconstruction in Erasure-coded Storage Systems
01h 14mErasure Coding at Scale
15mLESS is More for I/O-Efficient Repairs in Erasure-Coded Storage
18mDayu: Fast and Low-interference Data Recovery in Very-large Storage Systems
29mAccelerating Filesystem Checking and Repair with pFSCK