Horus: Granular In-Network Task Scheduler for Cloud Datacenters
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
A distributed in-network scheduler that runs entirely in the data plane and is implemented in P4 on Intel Tofino switches, Horus tracks per-worker load with compact registers and a lazy aggregated state updates scheme, resubmitting packets only when a provable drift condition is met. Evaluated on a testbed and in simulations with ~30k hosts, it reports 70–85% tail reduction and up to 5x throughput versus hierarchical RexKit, while cutting resubmissions and control-plane updates by orders of magnitude.
AI-written summary · about
chapters (10)
- 0:00Introduction to Horus
- 1:22Task scheduling challenges
- 3:24In-network scheduling
- 5:13Horus design principles
- 6:43Scheduling policy
- 8:15Packet processing
- 11:21State distribution
- 12:13Evaluation setup
- 13:52Testbed results
- 15:24Simulation and conclusion
related talks
16mTowards Optimal Rack-scale μs-level CPU Scheduling through In-Network Workload...
17mThemis: Scheduling-Aware Buffer Management for HBM-Based Hybrid Buffers
16mRingLeader: Efficiently Offloading Intra-Server Orchestration to NICs
16mHolistic and Automated Task Scheduling for Distributed LSM-tree-based Storage
46mIntro: Scheduling SIG
20mAdvancing HPC The Hybrid Scheduler Approach at the D.E.Shaw Group
24mEnhancing Homa Linux for Efficient RPC Transportation
16mGearbox: A Hierarchical Packet Scheduler for Approximate Weighted Fair Queuing