Holistic and Automated Task Scheduling for Distributed LSM-tree-based Storage
summary
HATS is a scheduler implemented atop Cassandra that co-schedules reads and compactions, using coarse-grained replica selection (epochal 60s global balancing via augmented gossip and a small Raft-backed scheduler), fine-grained per-request rerouting (dynamic snitch score) and adaptive rate control for compaction (compaction rate proportional to read load after replica decoupling). A linear-time assignment algorithm runs once per epoch; evaluated with YCSB and modeled Facebook traces, HATS cuts 99th-percentile latency by ~58–60% and raises throughput.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:48Distributed LSM-tree challenges
- 3:55Load balancing failures
- 6:01Compaction vs read performance
- 7:04HASS framework overview
- 8:08Read task assignment
- 12:24Compaction scheduling in HASS
- 13:27HASS performance gains
- 14:49Conclusion
related talks
01h 01mHTAP Workloads: Challenges and Solutions
30mScaling HDFS with Consistent Reads from Standby Replicas
18mHorus: Granular In-Network Task Scheduler for Cloud Datacenters
45mEasy Scaling with Open Source Data Structures
34mWhat We Talk About When We Talk About Disk IO
12mHotRAP: Hot Record Retention and Promotion for LSM-trees with Tiered Storage
22mELECT: Enabling Erasure Coding Tiering for LSM-tree-based Storage
20mFine Grained Replicated State Machines for a Cluster Storage System