Practice of Building AI Training Cluster Based on Kubernetes+RoCEv2
Talkat KubeCon + CloudNativeCon + Open Source Summit China 2023 · on Wednesday, 2023-09-27· at 15:50· 2层 会议室 2 | 2F Room 2·
summary
Covers practical solutions for running RDMA-based AI training on Kubernetes using RoCEv2 (UDP-based) instead of Infiniband. Describes Kubernetes network integration for lossless RoCEv2, pod-level access to RoCE NICs (including NIC virtualization), cluster network and lossless configuration, and resource scheduling/optimization for nodes with multiple GPUs and RoCE cards, plus required adjustments to training jobs.
AI-written summary · about
related talks
25mHigh Performance Networking for Distributed DL Training in Production K8s
34mNetworking Optimizations for Multi-Node Deep Learning on Kubernetes
39mScalable Multi-Node AI Workloads in Multi-Tenant AI Clouds U...
34mNetworking Optimizations for Multi-Node Deep Learning on Kubernetes
41mPanel: AI Infra Best Practices: Enterprise Do’s and Don’ts
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
36mMulti-Cloud Machine Learning Data and Workflow with Kubernetes
19mUsing Rook in Production on National Research Platform