VectorCDC: Accelerating Data Deduplication with Vector Instructions
summary
Presents a concrete method that rewrites hashless CDC algorithms into vector-friendly phases—tree-based extreme-byte search and packed range scans—implemented with SSE/AVX instructions. Shows measured throughput: VRAM (vectorized RAM) reaches ~25 Gbps and delivers ~16× speedup over unaccelerated RAM and 21× higher throughput versus prior vector work (uploader cites up to 46×); earlier AVX-512 SSCDC only achieves ~1.2–1.8×. Claims no loss in deduplication space savings.
AI-written summary · about
chapters (6)
- 0:00Introduction and overview
- 2:06Motivation for vector CDC
- 4:22Technical background
- 6:22Design of vector CDC
- 8:44Performance evaluation
- 11:11Conclusion and summary
related talks
24mAdvanced SIMD Algorithms in Pictures
01h 01mFaster, Easier, Simpler Vectors
54mFast C++ by using SIMD Types with Generic Lambdas and Filters
18mDistVS: Large-scale Vector Search with Compute-Memory Disaggregation
20mBoosting Video Codec With RISC-V Vector Extension
01h 02mSqueezing the Hardware to Make Performance Juice
01h 00mCache Friendly C++
01h 00mSIMD in C++20: EVE of a new Era