Dynamic Scheduling of Approximate Telemetry Queries
Talkat 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22) · 2022-04-04..04-06·
summary
System reframes telemetry as a runtime resource scheduler and implements a time-division approximation with cluster sampling to estimate error from only observed data. An optimization-driven scheduler targets per-query accuracy and latency while minimizing collector load and switch memory. Prototype DynATOS runs on Broadcom BroadScan hardware, reprogramming filters, hashing, and exports between sub-epochs. On a Maui Lab trace with an order-of-magnitude jump in sources, sketches drop to accuracy <0.5 while DynATOS adapts and meets goals.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:34Telemetry for Traffic Monitoring
- 2:35Illustrative Example
- 4:56Key Requirements
- 6:01Related Works
- 6:57Design Challenges
- 8:00Our Approach - DynATOS
- 8:55Key Insights
- 9:54(1) Time-Division Approximation
- 11:41(2) Scheduling Formulation
- 13:27Implementation on Switch Hardware
- 14:14Evaluation
- 14:50Robustness of Time-Division Approx.
- 15:38Tradeoffs Compared with Sketches
- 16:16Summary
- 16:42Future Work
related talks
17mPrecise Data Center Traffic Engineering with Constrained Hardware Resources
25mVirtualizing Traffic Shapers for Practical Resource Allocation
13mCognitive and Self-Adaptive System for Effective Distributed-Tracing
25mGuaranteeing Timeliness Via Prices...
33mReal-World Sampling – Lessons Learned...
17mKnown Knowns and Unknowns: Near-realtime Earth Observation Via Query Bifurcation in...
19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning Training
11mCognitive and Self-Adaptive System for Effective Distributed-Tracing...