Refurbish Your Training Data: Reusing Partially Augmented Samples...
summary
Claims a pipeline change that splits augmentation into partial augmentation and final augmentation and caches partially augmented samples to cut CPU overhead. Implements Revamper (≈2,000 lines Python on PyTorch 1.6) with balanced eviction and cache-aware shuffle. Evaluated on ResNet‑50 / ImageNet: training throughput improves 1.03×–2.04× (1.59× at reuse 2, 2.04× at reuse 3) while validation accuracy stays comparable (≈0.05% drop); plain data echoing harms accuracy.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:23DNN Training Pipeline
- 2:41Overhead of Data Augmentation
- 3:31Existing Approach: Data Echoing
- 4:48Our Approach: Data Refurbishing
- 5:16Analysis on Sample Diversity
- 6:31Standard Training
- 7:40Challenge: Inconsistent Batch Time
- 8:20PyTorch Dataloader
- 8:43Revamper
- 9:43Balanced Eviction
- 10:42Cache-Aware Shuffle
- 11:32Implementation
- 11:48Evaluation: Environments
- 12:05Evaluation: Baselines
- 12:27Evaluation: Accuracy & Throughput
- 13:50Conclusion
related talks
18mPreparation Meets Opportunity: Enhancing Data Preprocessing for ML Training With Seneca
28mMaximize GPU Utilization for Model Training
28mPer-epoch Shuffling Data Loader: Mix It Up As You Train!
28m🧼 From GPU-poor to data-rich: data quality practices for LLM fine-tuning
14mHyCache: Hybrid Caching for Accelerating DNN Input Preprocessing Pipelines
23mHeterogeneous Pipelines as the Unlock for ML at Scale
29mAutomated Model Update & Evaluation
33mBuild Better AI with First-Party Data and Fine-Tuning