Leveraging DRA for Conquering ML Workload Challenges on...
summary
DRA with a GPU resource driver enables controlled GPU sharing across and within pods, lets a single node host multiple GPU models on one node, and supports arbitrary placement constraints for LLM inferencing on Kubernetes. The demo uses NVIDIA MIG-style profiles, shows commands to create and list MIG profiles, and demonstrates dynamic reconfiguration of GPU partitions and allocation for inference workloads without tearing down deployments.
AI-written summary · about
chapters (7)
- 0:00ML workload challenges
- 5:31GPU infrastructure for ML
- 11:56Single GPU model runs
- 15:55Dynamic resource allocation
- 22:53Deploying models with MIG
- 27:06Configuring MIG profiles
- 30:49Q&A
related talks
24mFit-to-Serve: How a New DRA Capability for Dynamic Device...
28mA Deep Dive on How To Leverage the NVIDIA GB200 for Ultra-Fast...
29mWhat Can I Get You? An Introduction to Dynamic Resource Allocation
30mKubernetes WG Device Management - GPUs, TPUs, NICs and More With DRA
30mGPU Sharing at CERN: Cutting the Cake Without Losing a Slice!
31mAI, CERN, and the Quest for GPU Custody: How CERN Leverages DRA for Efficient GPU Sharing
50mDynamic Resource Allocation - the path towards GA
16mAI + Kubernetes: What Beginners...