Kubernetes Love Machine Learning, Even on Private Cloud
summary
Kubernetes runs machine learning on private clouds by wiring hardware, drivers, runtimes and device plugin plumbing into cluster automation and by using node labels and the resource API to classify GPUs. Major pain points are supporting heterogeneous GPUs (example: K80 vs newer cards), complex topologies like RDMA and NVLINK, and high contention because Kubernetes cannot request fractional GPUs so pods grab whole devices. Practical fixes include node labeling, topology manager, CPU manager and hugepages, vSphere/cloud-provider integrations and richer GPU metrics.
AI-written summary · about
related talks
31mKubernetes For GPU Powered Machine Learning Workloads In Academia
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
46mMachine Learning with Kubernetes
30mExplain How Kubernetes Works With GPU Like I’m 5
32mScaling Kubernetes Clusters for Generative Models: Managing GPU Resources for AI App...
26mMachine Learning Meets K8s
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
38mAll You Need to Know to Build Your GPU Machine Learning Cloud [B]