How we learned to love the Data Center Operating System
summary
Runs production services as Docker on Mesos at real scale — handling about 1 million bid requests/sec and large daily data ingest — using Marathon for long‑running apps and an in‑house simple scheduler for batch. Solves config with environment variables, hits cgroup pitfalls (containers see host RAM, OOM kills; CPU uses shares not absolute cores), uses Marathon‑lb and low DNS TTL for discovery, and attaches persistent volumes via a plugin (REX‑Ray → ScaleIO). Explores per‑container networking like Project Calico for routable IPs.
AI-written summary · about
chapters (15)
- 0:00Introduction to DCOS
- 2:37Early data science setup
- 6:00First container experiments
- 8:58Docker container basics
- 10:08Environment variables in containers
- 14:07Moving to Mesos framework
- 17:23Mesos architecture explained
- 19:38Introducing Sprint framework
- 21:09Resource allocation challenges
- 24:19CPU and network cgroups
- 27:18Networking in containers
- 29:05Evolution to microservices
- 33:36Persistent storage solutions
- 36:30Future plans for DCOS
- 37:39Production experience summary
related talks
53mDocker as the Building Block for Datacenter-Scale Applications
48mContainer: check! Aber wohin mit Persistent Data, Big Data oder Fast Data?!
19mKeynote: Operating 600+ Mesos Servers on 7 Datacenters @Criteo
27mDowntime is not an option
24mFrom VMs To Containers: A DevOps Journey
52mLessons Learned from building out hyper scale cloud services using docker
42mUsing Mesos to Drive Devops Adoption at Scale
42mMesos: The Operating System for your Cluster