Millisampler: Fine-Grained Network Traffic Analysis

on YouTube ↗21m

" summary

Millisampler is a lightweight eBPF-based profiler that samples per-CPU traffic at 100µs–10ms granularity by running a TC filter in kernel space and a user-space scheduler, storing ~2,000 samples per collection and estimating active flows with a 128-bit sketch. Deployed fleet-wide at low CPU and storage cost and optionally synchronized across rack hosts as sync millisampler, it uncovers microbursts that caused rack contention, exposed a vendor NIC firmware bug, and shows ~13% of racks highly contended with ~60% of tasks collocated.

AI-written summary · about

" chapters (4)

1. 0:00Start
2. 1:05Investigation Story
3. 5:21Requirement
4. 15:10Service Placement

" related talks

• 17mHow to diagnose nanosecond network latencies in rich end-host stacksLalith Suresh, Roni Haecki, Radhika Niranjan Mysore, Gerd Zellweger, Bo Gan, Timothy Merrifield, Sujata Banerjee, Timothy Roscoe · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 44mAdvanced Linux Kernel Networking MonitoringJizhong Jiang, Shane Xie · SREcon22 Asia/Pacific
• 36mThe black art of network performancesPaolo Abeni, Marcelo Ricardo Leitner · DevConf.CZ 2023
• 25mStroboscope: Declarative Network Monitoring on a BudgetLaurent Vanbever, Tobias Bühler, Ingmar Poese, Stefano Vissicchio, Olivier Tilmans · NSDI '18
• 17mSBB: Eliminating Centralized Bottlenecks in Userspace Network RuntimeRan Yi, Diyu Zhou, Jie Zhang, Zhenlin Wang, Xiaolin Wang, Yingwei Luo, Chuandong Li, Kang Hu, Shuqi Dong, Zonghao Zhang, Yiming Yao, Bo An · 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26)
• 46mSelf-Programming Networks: Network Clock Synchronization, Event Playback and Real-time ActionBalaji Prabhakar · Real-Time Decision Making 2018
• 16mClosed-loop Network Performance Monitoring and Diagnosis with SpiderMonWeitao Wang, T. S. Eugene Ng, Ang Chen, Xinyu Crystal Wu, Praveen Tammana · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 15mUnderstanding the impact of host networking elements on traffic burstsSoudeh Ghorbani, Erfan Sharafzadeh, Sepehr Abdous · NSDI 2023