Deploying and Scaling Spark ML and Tensorflow AI Models
summary
Shows a working pipeline that trains, packages and serves models with Tensorflow, Spark ML, Jupyter, Docker and Kubernetes plus NetflixOSS microservices, with code and images on pipeline.io. Demonstrates 100% open source artifacts, automated builds via Airflow hooks, Docker images per commit, and a canary model deployment that runs thousand-plus servers side-by-side for comparison. Covers model export (freeze_graph), graph transforms and quantization, request batching to increase GPU throughput, and metrics/circuit breakers that proved useful in cloud cost and latency tests.
AI-written summary · about
chapters (35)
- 0:00Intro
- 0:17Agenda
- 0:50Chris Fregly
- 1:53Pancake Stack
- 2:56High Performance Sensor Flow
- 3:43Spark Meetup
- 4:10IBM Spark Tech Center
- 4:53Tools
- 5:26Training Cluster
- 6:37Airflow
- 8:04Airflow Operators
- 8:29Triggers
- 9:49Airflow is Critical
- 11:21Tensorflow Example
- 11:40Code Example
- 12:20Celery and Flower
- 13:47Canary
- 16:11Continuous Deployment
- 17:06Example
- 17:43Metrics
- 18:54Price per prediction
- 20:20High availability
- 20:39Prediction cluster
- 21:52Prediction cluster on Amazon
- 22:42Different ways to deploy
- 23:38Tensorflow serving
- 24:04Generic keyvalue pairs
- 24:33Tensorflow is a neural network system
- 24:56Yappy FML
- 25:40Freeze Graph
- 27:16Build a new docker image
- 27:37Graph transform tool
- 28:35Pete Worden
- 29:20Latency Trading
- 31:22Rally Jupiter Notebook
related talks
30mDeploy end to end ML pipeline using Apache spark ...
58mThe path between developing and serving machine learning models.
35mBuilding Cross-Cloud ML Pipelines with Kubeflow with Spark & Tensorflow
48mDeploying Machine Learning Models in Practice
39mOperating Deep Learning Pipelines Anywhere Using Kubeflow
46mDeploying scalable Machine Learning models with Kubernetes
04mInterview
57mDevOps for Machine Learning