How We Built a Highly Scalable Machine Learning Platform Using Apache Mesos
summary
Built a containerized Machine Translation platform on Apache Mesos using Marathon/Kronos, Docker and Marathon for orchestration, Kafka for messaging, HBase on HDFS for storage, protocol buffers for messages and Ansible for automation, deployed to AWS. The system auto-scales microservices and reaches about 1,000 requests/sec and tens-to-hundreds of thousands of words/sec in the demo; lessons include Kafka/HBase/HDFS tradeoffs, 40% non-prod AWS spend and ~30% savings from reserved instances, and reliable deployments via Ansible + Docker + Marathon.
AI-written summary · about
chapters (21)
- 0:00Intro
- 1:46Machine Translation quality improvements
- 3:57Adaptive Machine Translation idea
- 5:21Update Example - EngFra
- 5:47Update Example: Translation Model Adaptation
- 9:19Neural MT in the Cloud
- 10:16How can we do this?
- 10:32What we had before?
- 13:05A new platform
- 14:50Architecture evolution
- 19:05Lessons Learned
- 19:19Cost efficient
- 23:52Security
- 25:41Platform high availability
- 28:11Resource allocation
- 30:58Releases are not as easy as expected
- 33:24Investigations become more complex
- 36:01Independent microservices
- 37:34Periodically reevaluate assumptions
- 39:36Future improvements
- 43:57Demo Time!
related talks
53mDocker as the Building Block for Datacenter-Scale Applications
01h 00mDeveloping Distributed Systems with Apache Mesos
40mHow to leverage the Apache Kafka Ecosystem to productionize Machine Learning
42mMesos: The Operating System for your Cluster
36mLessons Learned Running IBM Watson on Mesos
28mProductionizing Machine Learning Models at Scale with Kubernetes
43mLife of a startup...
57mMachine Learning Data Pipelines