Fast Vector Query Processing for Large Datasets Beyond GPU Memory with Reordered...
Talkat 21st USENIX Symposium on Networked Systems Design and Implementation (NSDI '24) · 2024-04-16..04-18·
summary
RUMMY is a GPU-backed vector query engine that pipelines host-to-GPU transfers and computation to handle billion-scale datasets beyond GPU RAM. It implements reordered pipelining, cluster-based retrofitting, and dynamic kernel padding with cluster balancing plus GPU-aware memory management. Built as a C++/CUDA system tested on NVIDIA A100, it reports up to 135× speedup over IVF-GPU with CUDA unified memory, up to 23.1× throughput vs a 64‑vCPU CPU baseline, and up to 37.7× better cost‑effectiveness on billion‑vector benchmarks.
AI-written summary · about
chapters (11)
- 0:00Vector query processing
- 1:49Vector queries in real-world apps
- 3:35Vector search challenges
- 5:24GPU limits for vector search
- 6:28Expanding GPU memory challenges
- 9:21Class-based query planning
- 10:25Computation optimization
- 11:29Pipeline design for transmission
- 13:15Ramy system overview
- 14:20Performance evaluation
- 15:04Conclusion
related talks
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
24mMoving to RISC-V Vector: A Practical Journey of AI Operator Optimization
01h 08mGPU Data Science with Rapids
41mAccelerating Data Science with RAPIDS
21mESPN: Memory-Efficient Multi-Vector Information Retrieval
35mBuilding High Performance Search Indexes in Rust with SIMD
16mAccelerate HPC and AI Applications with RVV Auto-Vectorization
40mUnder the hood of vector search with JVector