Building Distributed Systems with Apache Mesos

on YouTube ↗59m

" summary

Cluster management claim: Mesos (originating at Berkeley, in use at Twitter since early 2010) exposes a syscall-like API and implements two-level scheduling so different schedulers (Marathon, Kubernetes, Aurora, Yarn-on-Mesos) share one cluster and run stateful services like HDFS, Cassandra, MySQL, Redis, Kafka and Spark. It provides maintenance signals, volume and reservation primitives, higher utilization vs static partitions (reclaiming otherwise idle capacity), and supports rolling upgrades and high availability — frameworks run and survive node, agent, or master failures.

AI-written summary · about

" chapters (7)

1. 0:00Introduction and speaker background
2. 1:02What are data center applications
3. 8:00Challenges in running distributed systems
4. 19:36Introducing Apache Mesos as a cluster manager
5. 25:14Mesos as a data center kernel
6. 35:03Building vs running distributed systems on Mesos
7. 44:33Conclusion and Q&A

" related talks

• 42mMesos: The Operating System for your ClusterDavid Greenberg · Strange Loop 2014
• 01h 00mDeveloping Distributed Systems with Apache MesosYaniv Rodenski · NDC Sydney 2016
• 45mMesos, Then and NowBen Hindman, Ian Downes · Scale By the Bay 2017
• 57mA Distributed and Fault-Tolerant Application with Apache Mesos & Apache AuroraFlorian Pfeiffer · FrOSCon 2015
• 53mDocker as the Building Block for Datacenter-Scale ApplicationsBen Hindman · GOTO Amsterdam 2015
• 42mStreaming Data Pipelines on Apache Mesos: Lessons LearnedDean Wampler · MesosCon North America 2017
• 22mKeynote: The Future of Apache Mesos and DC/OSYrieix Garnier · MesosCon Europe 2017
• 27mKeynote Panel: Future of Cluster ManagementElizabeth K. Joseph · MesosCon North America 2017