How Booking.com serves Deep Learning Model Predictions

on YouTube ↗35m

" summary

Booking.com deploys Python deep learning models by training inside Docker containers on Hadoop data, saving TensorFlow checkpoints, and spinning up containerized Python apps that load model defs plus weights to serve predictions behind a load balancer. Kubernetes orchestrates scaling and rolling updates, GPUs attach to containers, and optimizations include quantization, freezing the graph and batching to trade latency for throughput — demonstrated at 1.2 million room-nights scale.

AI-written summary · about

" chapters (15)

1. 0:00Introduction
2. 1:21Agenda
3. 3:02The scale
4. 3:49Image tagging
5. 6:15How to use this information
6. 6:37Recommendation problem
7. 8:26Lifecycle of a model
8. 10:39Containers
9. 15:36Container Orchestration
10. 18:17Measuring Performance
11. 20:03Optimization
12. 23:34Optimizing for throughput
13. 24:26Batch requests
14. 25:57Summary
15. 27:11QA

" related talks

• 17mKeynote: Scaling Deep Learning Models in Production Using KubernetesSahil Dua · KubeCon + CloudNativeCon Europe 2018
• 33mDeep Learning at Booking.comEmrah Tasli, Stas Girkin · PyData Amsterdam 2017
• 40mMachine LearningVictor Bilyk · GDG DevFest Belarus 2018
• 44mMachine learning @ booking.comViktor Bilyk · HighLoad++ Siberia 2018
• 41mRESTful Machine Learning with Flask and TensorFlow ServingCarlo Mazzaferro · PyData DC 2018
• 17mLaying the foundation for deep learningMike Curtis · Collision 2017
• 22mThe Containerization of Machine Learning: TensorFlow: Kubernetes, and KubeflowSyed Ahmed · Montréal Cloud Native Day 2019
• 27mDeveloping and Deploying Models at Scale — without Spark!Eduardo Ariño de la Rubia · AI By the Bay 2017