Trip Down Memory Lane: OOM in Python
summary
Investigation shows a genuine Python memory leak: lists jump from 24k to 13 million and a process hits RSS ~2GB before swapping, causing system-wide slowdown. Repro tools are the resource module for getrusage, objgraph/heapy for object counts and referrers; garbage collection analysis proves live references block cleanup. The fix removes unneeded objects loaded into the hub; operational workarounds shard collectors and terminate nodes before the OS OOM killer runs.
AI-written summary · about
chapters (4)
related talks
25mIs it me or Python memory management? 🤔💻
23mMemoirs of a Python Object: Memory Management and Improving Performance
28mMemory management in CPython, fast or slow?
27mWhen Linux Memory Accounting Goes Wrong
28mKnocking On A Process’s Door: Introspecting Live Python Processes
33mMemory Profiling for Mere Mortals
45mCatching the 137-Killer: A Java Memory Forensics Investigation
01h 18mPython 103: Mmmm... Understanding Python's Memory Model, Mutability, and Methods