SNARY: A High-Performance and Generic SmartNIC-accelerated Retrieval System
summary
SNARY implements a SmartNIC-accelerated retrieval engine using on-card HBM and a data-parallel exact search plus an LSH-based fuzzy search, deployed on Xilinx commercial SmartNICs. It adds a parallel swap top-K algorithm that uses O(K) on-chip memory and early dropout to eliminate Top-K as a bottleneck. Measured on multi-million corpora, SNARY cuts exact-search latency 20.91%–83.88% and fuzzy-search latency 85.13%–87.40%, with throughput gains up to 18.27× (exact) and 23.81× (fuzzy).
AI-written summary · about
chapters (13)
- 0:00SNARY introduction and goals
- 1:20Industrial retrieval bottleneck
- 2:25Embedding-based retrieval needs
- 3:30Why FPGA and SmartNIC
- 4:35SNARY architecture
- 5:39Exact search implementation
- 7:08Parallel top-K algorithm
- 8:12Fuzzy search with LSH
- 9:17SNARY setup and evaluation
- 10:00Exact search performance
- 10:44Fuzzy search trade-offs
- 11:47LSH parameters and flexibility
- 12:11Summary and conclusion
related talks
17mSRNIC: A Scalable Architecture for RDMA NICs
32mRetrieval Augmentation and Semantic Search at Scale
18mScalable Billion-point Approximate Nearest Neighbor Search Using SmartSSDs
16mTowards High-throughput and Low-latency Billion-scale Vector Search via CPU/GPU...
32mUnder the Hood: Key Algorithms Powering Vector Search
17mDrs.NAS: Ultra-Efficient Neural Architecture Search for Recommendation Systems
30mSemantic Search and Neural Information Retrieval
36mWhen Semantic Search Breaks in your RAG systems: RAM Walls, Silent Failures, and the Architecture Decisions