Lightning Talk: Energy-Efficient Deep Learning with PyTorch and Zeus
summary
Zeus integrates with PyTorch to measure GPU time and energy via a Zeus monitor, and automatically tunes GPU knobs (power limit and SM frequency) with a power limit Optimizer that stays transparent to the ML code. Profiling across pipeline-parallel GPT-3 uncovers idle slack and yields 24% less energy with no training slowdown; the optimizer also traces the time–energy frontier to trade a little extra time for larger savings. Low-overhead measurements power a live LLM demo and plans include AMD support.
AI-written summary · about
chapters (12)
- 0:00Energy-efficient deep learning
- 0:48Energy efficiency in deep learning
- 1:30GPU energy consumption knobs
- 2:14Zeus Monitor API
- 3:17Power limit optimization with Zeus
- 3:59Pipeline-parallel energy optimization
- 6:28Time-energy trade-offs with Zeus
- 7:33Real-time energy for LLM chat
- 8:13Zeus framework principles
- 9:22Energy vs. carbon emissions
- 10:58Inference limitations and plans
- 12:09Hardware lifespan and carbon
related talks
15mZeus: Understanding and Optimizing GPU Energy Consumption of DNN Training
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
17mLightning Thunder: Supercharged PyTorch for Modern Hardware
08mSponsored Session: Lightning Talk: PyTorch in Production: Boosting LLM Training and...
07mLightning Talk: Environmentally Sustainable AI via Power-Aware Batch Scheduling
25mDistributed Training in PyTorch: Zero to Hero
05mLightning Talk: Running Energy-Aware AI Inference on Edge Kubernetes With Kepler
08mLightning Talk: Achieving SOTA GEMM Performance: A CuTeDSL Backend for PyTorch Induc...