Beyond malloc efficiency to fleet efficiency: a hugepage-aware memory allocator

on YouTube ↗14m

" summary

An enhancement of TCMALLOC called TEMERAIRE implements a huge-filler bin-packer that allocates in hugepage-sized spans, tracks per-hugepage metadata (longest free run, allocation count, pages used), and favors reducing fragmentation over best-fit to keep hugepages reclaimable. Evaluation on 8 apps shows +7.7% RPS and −2.4% RAM; a 1% fleet experiment and rollout yields ~6% fewer TLB-miss stalls, ~26% less memory wasted to fragmentation, about 1.3% fewer cycles stalled, and roughly 3–4% fleet RAM saved.

AI-written summary · about

" chapters (23)

1. 0:00Intro
2. 0:25normally reducing malloc cycles is good
3. 0:35business logic changes, infrastructure does not
4. 1:05malloc cycles do not matter. $$$ spent on hardware matter.
5. 1:35most cpu cycles do nothing
6. 2:08hugepages cheapen the page table walk
7. 2:48space efficient hugepage aware allocators are hard
8. 3:07demand oscillates wildly
9. 3:22emptying density - binpacking
10. 3:45mistakes can live forever
11. 4:14tcmalloc structure
12. 4:43spans back everything
13. 5:23change nothing but the page heap
14. 6:08Temeraire: the design
15. 6:34slack and donation
16. 7:05how does the HugeFiller make decisions?
17. 8:04HugeFiller tracks metadata per hugepage
18. 9:55we favor fragmentation over fullness
19. 11:06results
20. 11:21staged rollout
21. 11:52saved -1.3% of cycles
22. 12:06saving memory in the process
23. 12:32virtuous cycles: hugepage coverage