Greening the AI Cloud: Validating Power Models for Kubernetes Containers
Talkat Open Source Summit Europe 2023 · on Thursday, 2023-09-21· at 12:50· Room 5HTERRAZA (Floor 5), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Machine learning drives up to 40% of data center energy and Kepler framework measures per-process, container, pod and job power using eBPF collectors for low overhead telemetry, motherboard sensors on bare metal, and pre-trained VM power models built with regression when sensors are unavailable. Kepler runs as a CNCF Sandbox project, exposes Prometheus/Grafana metrics, partitions CPU/GPU/DRAM power (GPU driver readouts when available), and validates model estimates against node sensors showing closely matching results in MLPerf inference and training runs.
AI-written summary · about
chapters (14)
- 0:00Energy measurement in cloud apps
- 1:02Motivation for AI workload energy
- 5:45Kepler project overview
- 7:13Kepler architecture and power model
- 10:11Bare metal vs VM deployments
- 13:21Power model limitations
- 14:25GPU energy measurement
- 16:11MLPerf benchmark tool
- 18:19Kepler on Kubernetes
- 21:27MLPerf inference with Kepler
- 22:29Kepler Grafana dashboard
- 23:33Validating Kepler power measurements
- 24:56Summary and references
- 25:18Q&A on cloud energy
related talks
20mKepler - Optimized power usage monitoring for Kubernetes
27mHarnessing Kubernetes for Scalable AI/ML Workloads
12mKeynote: Cloud Native Sustainability - How You Can Help Right Now!
31mAccelerate AI adoption: automated cloud efficiency
06mKubernetes Love Machine Learning, Even on Private Cloud
19mResource Management Innovation for AI Workloads in Cloud Computing
40mPanel: Kubernetes and GPU Trends for AI in Financial Services
54mEconomics and Best Practices of Running AI/ML Workloads on Kubernetes