Putting Tensorflow Models into Production
summary
Shows how to go from a trained artifact to a running endpoint: export a saved_model, wrap it in Docker, and register it with TensorFlow Serving so the loader, dynamic manager and version policy handle memory, blue/green and gradual rollouts. Demonstrates HTTP and gRPC endpoints, warm‑up requests to avoid cold‑start p99 spikes, reuse of models from TensorFlow Hub, and simple examples (fashion MNIST) with measured latencies around 50–60 ms and a continuous training/deployment loop.
AI-written summary · about
chapters (18)
- 0:00TensorFlow model deployment intro
- 2:06Focus on deployment not training
- 3:35TensorFlow production advantages
- 8:59Challenges of TensorFlow
- 12:07ML deployment as engineering
- 15:58TensorFlow Hub pre-trained models
- 18:10Model Serving deep dive
- 29:56Docker setup for serving
- 37:11SavedModel format serving
- 43:04Model server with warm-up
- 48:12Local REST API predictions
- 51:59TensorFlow Hub in production
- 56:38TensorFlow Datasets for training
- 57:40TensorBoard for model analysis
- 1:02:32Text classification with TensorFlow
- 1:03:55Async client requests
- 1:06:21Model warm-up strategies
- 1:12:33Future of TensorFlow deployment
related talks
01h 18mPutting Tensor Flow Models Into Production
38mThe Hitchhiker's Guide to Tensorflow: from Tensors to running models in production
42mDeploying ML at Scale with Kubernetes and TFX
35mGetting Machine Learning Models Ready For Production Using Python
41mTensorFlow ever higher
41mRESTful Machine Learning with Flask and TensorFlow Serving
37mSimplifying Training Deep & Serving Learning Models...using Tensorflow
43mDeep learning beyond the learning