Ekya: Continuous Learning of Video Analytics Models on Edge Compute Servers
Talkat 19th USENIX Symposium on Networked Systems Design and Implementation (NSDI '22) · 2022-04-04..04-06·
summary
Edge servers run compressed video models that drift with live data; Ekya continuously retrains selected models and schedules GPU time to maximize mean inference accuracy. It uses a micro-profiler to predict accuracy-resource tradeoffs for hyperparameter configurations and a T-scheduler that allocates fine‑grained quanta of GPU time, implemented with Ray actors and NVIDIA MPS. Results show a 29% accuracy gain over a fair baseline and that the baseline needs ~4× more GPU resources to match Ekya.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:30Video data is everywhere
- 0:45Why video analytics at the edge?
- 1:23Edge Video Analytics Setup
- 3:48The cost of continuous learning
- 5:13Resource demands of continuous learn
- 6:21Summary thus far
- 7:15Scheduling decisions to make
- 7:57Working Example
- 8:34Example - Fair Scheduler
- 9:35Example - a smarter schedule
- 10:28Key Takeaways
- 10:51Ekya Design
- 12:05Ekya Thief Scheduler Goal: Maximize mean inference accuracy across all jobs
- 13:05Evaluation
- 13:35Scaling with increasing video streams
related talks
17mRECL: Responsive Resource-Efficient Continuous Learning for Video Analytics
16mGemel: Model Merging for Memory-Efficient, Real-Time Video Analytics at the Edge
01h 56mEdge AI Systems
50mAI acceleration at the edge
06mEdge AI in Practice: Lessons from Producti...
06mAI Inference at the Edge: Beyond the Hype
20mScaling Edge AI with Elixir and Nerves
48mThe Power of Edge AI: Revolutionizing Future of Real-Time, Autonomous Processing