BatchIt: Optimizing Message-Passing Allocators for Producer-Consumer Workloads
summary
Adds a small per-slab associative cache of remote frees that batches objects heading to the same slab, implemented for mimalloc and snmalloc implementations, so remote deallocations are queued in slab-local batches instead of atomically touching global structures. Experiments show reduced atomic operations and cache misses and >20% speedups on some producer‑consumer microbenchmarks. Costs include potential increased slab lifetime and fragmentation in mimalloc and a need to tune cache size and associativity for real workloads; batching of slab-directed frees.
AI-written summary · about
chapters (9)
- 0:00BatchIt intro and workloads
- 1:14Producer-consumer memory patterns
- 2:41Message-passing allocators background
- 4:49Thread caching and lock contention
- 6:14Malloc and SN Maloc design
- 8:43Malloc vs SN Maloc trade-offs
- 10:11BatchIt optimizations
- 12:20Benchmarking results
- 14:27Q&A
related talks
19mConcurrent Data Structures Made Easy
44mRestartable Sequences: Scheduler-Aware Scaling of Memory Use on Many-Core Systems
43mC++ Atomics...
01h 07mBuilding Robust Inter-Process Queues in C++
01h 08mDesigning fast lock free algorithms by understanding cache coherence dynamics
01h 32mLock-free programming with modern C++
26mTackling Contention: The Monsters Inside the 'sync.Locker'
22mBBQ: A Block-based Bounded Queue for Exchanging Data and Profiling