Preparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca
summary
Seneca is a modified PyTorch dataloader that uses model-driven partitioning to split DRAM between encoded, decoded, and augmented forms and opportunistic data sampling to prefer cached items during random batch draws. It models DSI throughput from hardware specs and measured preprocessing rates, chooses cache partitions at job start, and serves cached samples opportunistically. Compared to PyTorch/DALI/Shade/Quiver it cuts makespan ~45.23%, boosts data throughput up to 3.45×, and achieves ~54% cache hits with only 20% of the dataset cached.
AI-written summary · about
chapters (16)
- 0:00Seneca introduction
- 1:20Hardware challenges in preprocessing
- 2:23DSI pipeline challenges
- 3:44Caching trade-offs for training
- 5:11Storage bottlenecks
- 5:55Random sampling issues
- 6:59DSI performance modeling
- 8:45Validating DSI model
- 10:09Seneca’s MDP and ODS methods
- 10:51Evaluation setup
- 11:33Seneca’s impact on accuracy
- 12:39Performance gains with hardware
- 13:40Seneca scalability
- 15:04Why Seneca’s high cache hits
- 16:47Key results and contributions
- 17:09Summary
related talks
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
18mSHADE: Enable Fundamental Cacheability for Distributed Deep Learning Training
13mCombining algorithms and quality data to build better AI platforms
24mThe Evolving Landscape of Dataloading
24mFast training and near-linear scaling with DataParallel in Amazon SageMaker
15mRefurbish Your Training Data: Reusing Partially Augmented Samples...
18mTierTrain: Proactive Memory Tiering for CPU-Based DNN Training
28mFault Tolerance Training at Pinterest