Optimizing Knowledge Distillation Training With Volcano
summary
EDL deploys the teacher as an online inference service via Pilot Serving and uses EDL service distillation with Volcano scheduling and IO/RDMA awareness to place teacher instances on idle or fragmented GPUs (k40, P4) while training students on V100. Benchmarks on ImageNet (ResNet-50) show distillation raises accuracy ~2%, service distillation retains 82.8% throughput of pure training and runs 2.3× faster than same-network distillation; cluster utilization rises above 70%, queue time <44 minutes and job failure <5%.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 0:15Project Background
- 1:33Large Scale Training
- 2:35Elastic Deep Learning
- 3:47Knowledge Distillation
- 8:25Advantages
- 12:23Training Vector
- 14:15William Wang
- 15:37Challenges
- 18:10CNCF Sandbox
- 18:43Volcano Architecture
- 22:58Survival Kubernetes
- 24:42Volcano Job
- 26:06GPU Sharing
- 26:43Cromwell
- 27:05Commander
- 27:33Kubernetes
related talks
37mImproving Performance of Deep Learning Workloads With Volcano
35mCompress language models to effective & resource-saving models with knowledge distillation
08mLeveraging large-scale models for efficient learning
29mOptimizing Training Performance for Large Language Model(LLM) in Kubernetes
19mNo GPU Left Behind: Scaling Online LLM Training With...
22mSpecializing Small Language Models with less data
08mKeynote: Scaling Model Training with Volcano: iFlytek’s Kubernetes Breakthrough
36mAccelerating High-Performance Machine Learning at Scale i...