Offload Annotations: Bringing Heterogeneous Computing to Existing Libraries...
summary
Offload Annotations maps existing CPU libraries like NumPy, Pandas and scikit‑learn to GPU kernels (PyTorch, cuML) via small annotation files and a runtime that schedules execution, transfers and automatic paging of chunks that exceed GPU memory. An annotator typically writes about 130 lines per integration, the system is evaluated on eight real workloads and matches handwritten GPU ports while achieving up to 1200x and a median 6.3x speedup, including transparent data splitting and device-aware scheduling.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:24Background: Hardware Commoditization
- 0:45Background: CPUs vs. GPUs
- 1:46Background: Data Science on the CPU
- 2:16Trend: Data Science on the GPU
- 2:58Trend: CPU Libraries vs. GPU Libraries
- 6:16Goals
- 6:46Annotator - Function Annotations
- 8:02Step 1: Annotator - Allocation Annotations
- 8:28Step 1: Annotator - Offload Split Type
- 10:30End User
- 13:49Step 3: Runtime - Offloading API
- 14:19Step 3: Runtime - Splitting API
- 14:59Step 3: Runtime - Scheduling Heuristics (optional)
- 16:59Integration Experience
- 17:17Evaluation: Summary
- 19:42In-Depth Evaluation: Allocations
- 20:22In-Depth Evaluation: Heuristics
- 20:58In-Depth Evaluation: Splitting/Paging Datasets
- 22:24Conclusion
related talks
34mGPU as a Service Over K8s: Drive Productivity and Increase Utilization
36mAccelerating High-Performance Machine Learning at Scale i...
33mAI workloads on Kubernetes - How to maximize GPU utilization and cut costs
09m⚡ Lightning Talk: Advanced GPU-Orchestrated Workflows and HPC Integrations on K8s for Distributed AI/ML at Scale
04mLightning Talk: Dynamic GPU Fraction and Sharing With Cloud Native Principle
08mLightning Talk: GPU.x - Dynamic GPU Fraction and Sharing on ML Mainstream Framework
42mOpen Source is Better Together- GPU Python Libraries Unite
30mCUDA in Your Python: Effective Parallel Programming on the GPU