Declarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
summary
Hammy implements fractional GPUs by intercepting CUDA calls and running a custom device plugin, admission webhook, scheduler and ham-core; the demo assigns 4 GB memory and 30% compute to three pods on a single A100 (40 GB) — all three run and a fourth goes pending when capacity is exhausted. It maps to a resource claim model to decouple pods from hardware, contrasts with MIG/MPS/time-slicing, and requires workload benchmarking and vendor DRA drivers for production.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:25GPU demand in Kubernetes
- 4:33Existing GPU sharing strategies
- 10:36Hemi project for fractional GPU
- 16:23Hemi architecture features
- 21:46Dynamic GPU resource allocation
- 29:47Hemi vs DRA comparison
- 36:15Key takeaways
- 37:20Q&A
related talks
34mDeclarative Device Virtualization: Orchestrating GPUs & Hardware in Cloud Native Environments
05mHami: Dynamic, Smart, Stable GPU-Sharing Middleware In Kubern...
27mUnlocking Heterogeneous AI Infrastructure K8s Cluster: Leveraging the Power of HAMi
10mLightning Talk: K8s Issue #52757: Sharing GPUs Among Multiple Containers
31mShared GPU Scheduling & Proactive Autoscaling: A Production Blueprint for 1000+…
41mDynamic GPU Provisioning in Kubernetes With Composable Disaggregated...
30mCoHDI: A CNCF Sandbox Project for Dynamic Hardware Composability in Kubernetes
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!