Unlocking How To Efficiently, Flexibly, Manage and Schedule Seven AI Chips in Kubernetes
Talkat KubeCon + CloudNativeCon Europe 2025 · on Friday, 2025-04-04· at 14:30· Level 0 | ICC Capital Hall | Room J·
summary
Kubernetes middleware that manages seven different AI accelerators and provides device sharing via an injected library that hijacks CUDA calls (requires CUDA >=10.2 and driver >=440), a mutating webhook plus a topology-aware scheduler extender, and virtualization support for six vendors. Claims boost GPU utilization from ~50% to nearly 100% on shared nodes, exposes allocation vs real usage metrics, supports task priority and memory limits, and integrates with Volcano/Koordinator and Kueue; about 100 users and production adopters report success.
AI-written summary · about
chapters (10)
- 0:00Efficient AI chip management in Kubernetes
- 2:06Traditional GPU sharing challenges
- 3:33Limitations of Kubernetes DRA
- 5:01Introducing Hami for AI devices
- 6:47Hami GPU sharing and memory control
- 8:55Advanced scheduling in Hami
- 11:06Topology-aware scheduling policies
- 12:35Monitoring GPU with Hami metrics
- 14:23Hami adoption and roadmap
- 18:24Q&A
related talks
32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi- 32mScaling AI Workloads with Kubernetes: Sharing GPU Resources Across Multiple Containers
31mAccelerate AI adoption: automated cloud efficiency
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
24mEmpower Heterogeneous Edge AI Acceleration with K8s
20mHow are You Utilizing GPUs? Best Practices on Managing GPUs in Azure
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...