Compacting Heaps in Ruby 2.7

on YouTube ↗47m

" summary

Ruby 2.7 includes a manual heap compactor integrated with the GC and the talk explains exact memory layout, a two‑finger compaction algorithm, pinning bits, compaction callbacks and a rb_gc_location API for extensions. The patch is multi‑year work, initially pins about 3% pinned → 10% smaller heap on a Rails app and later drops to ~1% pinned; object_id shifts to a monotonic object_id. Prototype works but is slow, requires GC.compact today and aims for automatic compaction in Ruby 3.0.

AI-written summary · about

" chapters (29)

1. 0:00Start
2. 3:54Ruby Cores
3. 4:22Pipeline Operator
4. 4:41Smile Operator
5. 9:39Efficient Memory Usage
6. 10:35CPU Cache Hits
7. 14:18Fragmented Memory
8. 16:34Ruby's Heap Layout
9. 18:32Moving Objects
10. 19:46Updating References
11. 20:55Object Movement
12. 21:38Reference Updating
13. 22:25Finding References
14. 24:00Array
15. 24:34Hashes
16. 27:42Yajl Mark Function
17. 28:41Pinning Bits
18. 31:47Compaction Callback
19. 32:10Problem Object Graph
20. 33:35Compilation Process
21. 33:53RubyVM Instruction Sequence
22. 35:59Problem Code
23. 38:18object_id is based on location
24. 39:50Object ID Collisions
25. 40:12Collision Resolution
26. 40:47GC Cleanup
27. 43:24Basic Rails Application
28. 45:25Sliding Compaction
29. 45:45Fixed Width Layout

" related talks

• 51mCompacting GC for MRIAaron Patterson · Balkan Ruby 2019
• 54mKeynote: Exploring Memory in Ruby - Building a Compacting GCAaron Patterson · RubyConf MY 2017
• 48mDefragging RubyAaron Patterson · RubyConf AU 2017
• 51mCompacting GC in MRIAaron Patterson · RubyConf Colombia 2017
• 33mThe Bug that Forced Me to Understand Memory CompactionEmily Giurleo · RubyConf 2020
• 34mUnder the Hood of Ruby's Generational Garbage CollectorHemant Kumar · Rocky Mountain Ruby 2014
• 01h 01mKeynote: Mark/Compact GC in MRIAaron Patterson · Red Dot Ruby Conference 2017
• 45mClosing Keynote: Analyzing and Reducing Ruby Memory UsageAaron Patterson · GORUCO 2018