Towards Plan-aware Resource Allocation in Serverless Query Processing
summary
A plan-aware allocator called resource safer reads query plans and live job stats, converts the query DAG into disjoint trees with TreePhi, runs a fast one-pass DFS heuristic to approximate the max-vertex-cut per tree, and periodically shrinks resources if the remaining peak falls below allocation. A lightweight greedy implementation runs in linear time, is conservative to avoid starvation, and on 154 production SCOPE jobs yields per-job savings of 0–75% and 8.3% cumulative resource reduction (with up to 19% seen across clusters).
AI-written summary · about
chapters (9)
- 0:00Introduction
- 2:23Query Estimates
- 3:04Planaware Resource Allocation Strategy
- 7:41Resource Allocation Strategy
- 9:14Remaining Peak Algorithm
- 9:44Verification
- 11:44Calculation
- 14:00Experiment Results
- 15:04Summary
related talks
21mHarmonizing Efficiency and Practicability: Optimizing Resource Utilization in...
11mLightning Talk: Searching for Serverless Candidates in the Jungle of Microservices
51mAdvanced Plan Explorer Usage for Tuning Execution Plans
18mALPS: An Adaptive Learning, Priority OS Scheduler for Serverless Functions
18mSOPHIA: Online Reconfiguration of Clustered NoSQL Databases for Time-Varying Workloads
41mUsing Serverless & ARM64 for Real-Time Observability
26mShuffling, Fast and Slow: Scalable Analytics on Serverless Infrastructure
10mEnergy-Efficient and Cost-Optimized Serverless Computing