Solving inefficiency challenges at Datacenter scale
summary
An engineering team finds storage nodes using <20% CPU, then implements kernel isolation with cgroups and namespaces and socket pinning so storage and compute safely share one physical machine. The default node becomes co-location mode, flipped by a single configuration bit and rolled out across tens of thousands of servers. The result is about a 9% capacity boost and claimed 9-figure CAPEX savings, plus lower storage tail latencies.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 3:30Toil and SRE practices
- 5:15Data center scale at Twitter
- 8:24Capacity management challenges
- 10:50Compute platform architecture
- 13:19Storage platform architecture
- 15:44Underutilized CPU resources
- 18:10Co-location implementation
- 20:18Nine-figure capex savings
- 23:25Conclusion and key takeaways
related talks
27mThe Storage Architecture of Intel's Data Management Platform (DMP)
44mFinOps at scale: Lessons from scaling Kubernetes workloads
39mHow we learned to love the Data Center Operating System
30mWhen the Edge Can’t Afford a Third Node: A Storage Solution for Two-Node Kubernetes C...
27mPOWER your CLOUD
29mTowards Sustainable Multi-Cluster Management in Real-World Academic Infrastructure
25mContainer Attached Storage: Open Source Extensions: Kubernetes + Data
47mCloud Native and Sustainable, Reproducible Scientific Computing