Deploying Deep Learning models on Kubernetes clusters with GPU Enabled
summary
Deploying a Keras deep-learning model (TensorFlow backend) into a GPU-enabled Kubernetes cluster, comparing inference performance on GPU vs CPU, and exposing the model as a REST API for production use. The talk covers measuring inference speed differences and the steps to put the model into production on Kubernetes with GPU support.
AI-written summary · about
chapters (14)
- 0:00Model deployment introduction
- 1:07Model deployment basics
- 3:14Production deployment challenges
- 4:21Computer vision models
- 9:18GPU vs CPU for deep learning
- 11:44HTTP data transfer protocols
- 15:27Neural network architecture
- 26:43Architecture comparison
- 31:21Production infrastructure
- 34:34Development common steps
- 39:30Testing and accuracy requirements
- 41:57Amazon SageMaker
- 47:34Deployment options
- 50:01Closing and next steps
related talks
36mProduction GPU Cluster with K8s for AI and DL Workloads
37mBuilding Distributed TensorFlow Using Both GPU and CPU on Kubernetes [I]
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
27mHarnessing Kubernetes for Scalable AI/ML Workloads
17mKeynote: Scaling Deep Learning Models in Production Using Kubernetes
38mMinimizing GPU Cost for Your Deep Learning on Kubernetes
32mGPU Machine Learning From Laptop to Cloud
34mBuilding GPU-Accelerated Workflows with TensorFlow and Kubernetes [I]