Machine learning at scale using Kubeflow with Amazon EKS & Amazon EFS

on YouTube ↗54m

" summary

Shows how to run Kubeflow on Amazon EKS using the EFS CSI driver to mount shared storage, create a storage class, and run a TensorFlow training job that uses an EFS-mounted dataset; demo provisions the EFS file system, creates an access point, launches a Jupyter notebook, and runs a two-replica job that completes (training ends with ~52% accuracy for the small flowers dataset). Highlights EKS + Kubeflow + EFS, 500 TB customer example, and two TensorFlow replicas.

AI-written summary · about

" chapters (7)

1. 0:00ML with containers
2. 1:59ML on Kubernetes
3. 5:07Kubeflow and EFS overview
4. 7:24Deep dive into Amazon EFS
5. 16:05Architecture and setup
6. 21:24Hands-on demonstration
7. 43:00Resources and closing

" related talks

• 35mMachine Learning at Scale Using KubeFlow on AWSSiman Debnath · Kubernetes Community Days Chennai 2022
• 51mMachine Learning using Kubeflow and KubernetesArun Gupta · Devoxx Belgium 2019
• 18mKeynote: Containerizing Deep Learning Using Kubeflow on AWSSukanya Mandal · Kubernetes Forum Bengaluru 2020
• 46mAccelerating Machine Learning DevOps with KubeflowDerek Ferguson · GOTO Chicago 2019
• 40mStreet fighting techniques for multi tenant machine learning...Sky Thomas · Scale By The Bay 2018
• 34mBuilding End-to-End Machine Learning Workflows with Kubernetes...Antje Barth · Global Artificial Intelligence Virtual Conference Sep 2020
• 46mKubeflow, TensorFlow on KubernetesIsrael Olalla · Commit Conf 2018
• 46mMachine Learning with KubernetesJiaxin Shan, Yaniv Donenfeld · AWS Container Day 2019 San Diego