Distributed Network Monitoring and Debugging with SwitchPointer

on YouTube ↗21m

" summary

SwitchPointer stores only pointers in switches and keeps detailed telemetry at end-hosts, using switch memory as a "directory service" to locate per-epoch data. It uses a hierarchical epoch index (example: alpha=10, K=3 → 345 KB for 100k hosts and pushes top-level pointers at ~100 kbps), a minimal perfect hash (≈2.1 bits per host) to avoid collisions, and embeds linkID/epochID in packets. A software implementation on real testbeds debugs temporal and spatial cascades and shows no throughput loss for packets ≥256 B (22% drop at 128 B).

AI-written summary · about

" chapters (16)

1. 0:00Intro
2. 0:19Network monitoring and debugging is complex
3. 0:49An example: Too many red lights
4. 3:07Switch Pointer Integrates the best of two worlds
5. 3:59Switch Pointer in a nutshell Divides time into epochs
6. 5:06Let's revisit "Too many red lights" problem
7. 5:54Switch Pointer design
8. 9:09Minimal Perfect Hash Functions
9. 10:28Switch Pointer: Four technical challenges
10. 11:32Switch Pointer - Coverage
11. 12:35Problems Switch Pointer cannot debug
12. 12:49A more complex example: Traffic cascades
13. 14:53Switch Pointer overhead (software implementation)
14. 15:40Conclusion
15. 17:2515th USENIX Symposium on Networked Systems Design and implementation
16. 18:19THE ADVANCED COMPUTING SYSTEMS ASSOCIATION

" related talks

• 12mDebugging Transient Faults in Data Centers using Synchronized Network-wide PacketRaj Joshi, Pravein Govindan Kannan, Nishant Budhdev, Mun Choon Chan · NSDI ’21
• 16mClosed-loop Network Performance Monitoring and Diagnosis with SpiderMonWeitao Wang, T. S. Eugene Ng, Ang Chen, Xinyu Crystal Wu, Praveen Tammana · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 46mSelf-Programming Networks: Network Clock Synchronization, Event Playback and Real-time ActionBalaji Prabhakar · Real-Time Decision Making 2018
• 32mSnabb Switch simple and fast packet networkingLuke Gorrie · HighLoad++ 2015
• 16mIMap: Fast and Scalable In-Network Scanning with Programmable SwitchesMingwei Xu, Guanyu Li, Menghao Zhang, Cheng Guo, Han Bao, Hongxin Hu, Fenghua Li · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 32mSnabb Switch simple and fast packet networkingLuke Gorrie · HighLoad++ 2015
• 16mHeteroSketch: Coordinating Network-wide Monitoring in Heterogeneous and Dynamic NetworksAnup Agarwal, Srinivasan Seshan, Zaoxing Liu · 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22)
• 23mNetwork Communication Debuggability and Observability at ScaleAshmitha Jeevaraj Shetty, Min Si · Networking @Scale 2024