Thank you, next: Iterators
summary
Company runs an internal analytics platform called NAR that feeds Spark using Kinesis shard iterator instances forced into synchronous batches; Spark’s core execution follows a pull-based iterator (think hasNext/next) model that composes and fuses operators but proves fragile for async IO and resource cleanup. The claim: iterators are a compact, efficient abstraction across Scala, Kafka and Cassandra yet carry mutable-state pitfalls, and many iterator use-cases can be recast functionally with applicative functors or observer duals.
AI-written summary · about
chapters (8)
- 0:00Start
- 5:11Excerpts from the literature
- 8:50What are iterators - Different classifications
- 15:04C# interface for iterators
- 17:49Iterate any shape
- 23:01Scala iterators
- 23:55Transformation fusion
- 24:59Grammarly iterators
related talks
28mWhat happens next: a choose-your-own-adventure with iterators
27mA, B, C. 1, 2, 3. Iterables you and me
04mInterview
24mGenerating async iterators is ekstreamly easy, I Promise ;)
33mStream into the Future
51mThe Miracle of Generators
31mTwo fundamental abstractions
50mGenerators: A Hummer in the Stable?