Turbocharge ANNS on Real Processing-in-Memory by Enabling Fine-Grained Per-PIM...
summary
PIMANN retrofits UPMEM by tapping an undocumented per-PU control interface to switch DDR ownership and run a persistent PIM kernel that eliminates inter-batch idle time, plus per-PU query dispatching with cluster slicing and hotness-based replication to fix intra-batch imbalance. Implementation modifies the UPMEM driver and runs on SIFT1B and SPACE1B, raising PIM utilization from 18.2% to ~80% and boosting throughput by 2.4–10.4× while cutting average latency 32–43% and improving throughput-per-watt up to 2.5×.
AI-written summary · about
chapters (14)
- 0:00Kim Alen and PIM-based ANNS
- 1:19ANNS fundamentals and IBFPQ
- 3:11CPU/GPU ANNS limitations
- 4:15UPNM PIM hardware
- 5:07PIM batch scheduling bottlenecks
- 7:40Fine-grained per-PU scheduling
- 9:05System architecture and PIM kernel
- 10:10Persistent PIM kernel details
- 12:23Driver changes for per-PU control
- 13:29Per-PU query dispatching
- 15:18Evaluation setup and datasets
- 16:23PIM throughput improvements
- 17:33Latency and power efficiency
- 18:17Conclusion and open-source
related talks
01h 05mReal In-Memory Processing
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
06mTensorPRAM: Designing a Scalable Heterogeneous Deep Learning Accelerator
45mHigh-Performance and Scalable Middleware for HPC and Deep...
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
14mAIE-ML dev PMD for VRAN and UPF Datapath Acceleration
28mSimplifying AI for Edge Compute with HAMi
14mBrain-on-Switch: Towards Advanced Intelligent Network Data Plane via NN-Driven Traffic...