Muves: Multimodal & multilingual vector search w/ Hardware Acceleration
summary
Muves combines Elasticsearch/OpenSearch with a GSI APU backend to run multimodal multilingual vector search using CLIP and multilingual sentence transformers and a built‑in fine‑tuner. Demo indexes a 10M subset of LAION 400M and returns image vector results in ~69 ms and a 20‑query batch in 265 ms; prior benchmarking shows GSI at ~92 ms versus CPU solutions. Evaluation with Quepid reports image‑embedding NDCG@10 topping keyword and text embeddings (text ~75, keyword ~84 average), and pre‑filtering works via the APU plugin.
AI-written summary · about
chapters (10)
- 0:00Introduction to MUVS
- 2:15Vector search vs. keywords
- 5:12GSI hardware acceleration
- 6:50Architecting the solution
- 9:02MUVS components
- 11:09Implementation and workflow
- 14:34Live demo and performance
- 22:42Evaluation metrics
- 26:21Lessons learned
- 30:39Q&A session
related talks
43mcuVS and Lucene: GPU-based Vector Search
47mUnlocking GPU Powers for Vector Search: OpenSearch's New Frontier
31mFrom ML to LLM on device AI in the browser
33mTranslate tech talks using open-source models and cloud-native Wasm
20mFrom ML to LLM: On-Device AI in the Browser
15mMediaPipe Web: Bringing cross-platform AI tech to the browser
24mState of client side machine learning
01h 00mGPU Accelerated Computing & Optimizations on Cross-Vendor Graphics Cards with Vulkan & Kompute