Kareus: Joint Reduction of Dynamic and Static Energy in Large Model Training
Talkat 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26) · 2026-07-13..07-15·
summary
Kareus jointly optimizes GPU frequency, SM allocation and launch timing, and kernel scheduling using multi-objective Bayesian optimization per partition to push the time–energy frontier. It evaluates configs on 8-GPU A100 nodes across 14 workloads (Llama 3.2, Qwen 3), builds per-partition frontiers with a four-pass acquisition strategy, and applies CUDA grid sizes plus a frequency controller at runtime. Results claim up to 28.3% training energy reduction at equal time or up to 27.5% time reduction at equal energy versus prior work.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:00Energy breakdown in GPU training
- 2:25Limitations of existing approaches
- 3:47Case studies: kernel scheduling and frequency
- 7:22Key insight behind CARS
- 8:24Multi-objective Bayesian optimization
- 10:13System integration and runtime scheduling
- 12:43Performance evaluation results
- 14:30Future work and open-source
related talks
15mUSHER: Holistic Interference Avoidance for Resource Optimized ML Inference
36mAccelerating AI training fleets with sched_ext
15mZeus: Understanding and Optimizing GPU Energy Consumption of DNN Training
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
28mMaximize GPU Utilization for Model Training
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
34mDeep Customized Kubernetes for Machine Learning in Tencent
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...