Streaming data transformations with Ruby
summary
Implements a streaming pipeline that fetches a webpage, pipes it through a gzip decoder implemented as an Enumerator IO wrapper, and counts characters without loading the whole document into memory, returning about 132000 characters on the example page. Composition uses lambdas that return Enumerables and is packaged as the typewriter gem, with the pipeline logic reproducible using Ruby 2.7’s function composition operator; the demo produces identical output to the shell pipeline.
AI-written summary · about
related talks
09mPipe Operator for Ruby
34mBuilding Data Driven Products with Ruby
36mWorking with binary data in Ruby
26mBuilding Stream Processing Applications with Ruby & Meroxa
36mStreaming over the web with modern Ruby
33mEnumberable's Ugly Cousin
45mRuby for the soul of Big Data nerds
27mEach All the Way Down: How Ruby Enumerators Power Streaming...