Meta Generators: Playing Safe with Long Sequences
summary
Shows that using itertools functions like product can silently blow up memory because itertools.product materialises iterables so a 100-note combination step produces about 20 trillion combinations and forces buffering. Demonstrates practical use of range, combinations, chain and product, then builds meta-generators that add __iter__/copy/reversed to lazy iterables and a custom safe streaming product which preserves laziness and fixes the bug in real code, and it works for the reported use case.
AI-written summary · about
chapters (10)
- 0:00Start
- 1:50What is a Generator?
- 3:49Benefits of Generators
- 5:30Music Matching
- 15:03Clone the Generator, Right?
- 16:32Pass the Generator Function
- 18:21Generator Pattern
- 19:13Copyable Meta Generator
- 20:47Reversible Meta Generator
- 22:08Are Iterator Functions Safe?
related talks
33mA tour of the module itertools
32mIterating Efficiently in Python with itertools
29mPythonic iterators and generators
27mWhen to refactor your code into generators and how
03mReimplement `itertools` for fun & profit
49mGenerators Will Free Your Mind
01h 26mBeyond Python Enhanced Generators
01h 26mBeyond Python Enhanced Generators