From High Performance Computing To AI Workloads on Kubernetes: MPI Runtime in Kubeflow TrainJob

on YouTube ↗30m

" summary

Kubeflow TrainJob integrates an MPI runtime that runs MLX and DeepSpeed workloads on Kubernetes, exposing a Python Qflow SDK so data scientists call one function instead of writing manifests. Example runs a CNN with MLX over three nodes on 60,000 images using DDP and saves the model, and fine‑tunes T5 with DeepSpeed with 8 GPUs (two nodes × four GPUs) in ~69 seconds, exporting checkpoints to S3. It uses an SSH‑based MPI setup to avoid control‑plane load and provides MLX distributed runtime and runtime package inspection.

AI-written summary · about

" chapters (12)

1. 0:00AI workloads needs
2. 1:25Kubernetes for data scientists
3. 2:27CubeFlow Trainer project
4. 3:09Scaling MLX on Kubernetes
5. 7:03Fine-tuning T5 with DeepSpeed
6. 13:45Behind the demos
7. 16:14HPC to AI workloads
8. 18:26Migrating to Kubernetes
9. 22:46MPI and PyTorch solutions
10. 27:07CubeFlow Trainer MPI runtime
11. 29:01Conclusion and future work
12. 30:08Q&A

" related talks

• 46mMachine Learning with KubernetesJiaxin Shan, Yaniv Donenfeld · AWS Container Day 2019 San Diego
• 31mKubernetes for Machine Learning: Managing and Scaling AI WorkloadsDrishti Jain · KCD Mumbai 2023
• 51mMachine Learning using Kubeflow and KubernetesArun Gupta · Devoxx Belgium 2019
• 14mKubeflow: Where ML Meets KubernetesSuresh Peiris · KCD Sri Lanka 2023
• 54mEconomics and Best Practices of Running AI/ML Workloads on KubernetesMaulin Patel · KubeCon + CloudNativeCon Europe 2019
• 30mKubeflow, Your ML Tools on KubernetesLaura Morillo, Iván Cañaveral · Commit Conf 2018
• 54mScaling Machine Learning Pipelines in CloudSalman Iqbal · NDC Oslo 2021
• 39mOperating Deep Learning Pipelines Anywhere Using KubeflowJörg Schad, Gilbert Song · KubeCon + CloudNativeCon China 2018