GenStage and Flow
summary
GenStage implements a demand-driven back-pressure contract so producers only emit what consumers request (defaults: max_demand 1,000, min_demand tunes prefetch), and Flow is a concurrent layer built on GenStage. Flow codebase is small — Flow ≈1,200 LOC with extensive docs — and partitions by hash so identical keys route to the same reducer. Benchmarks show eager approaches stall, stream runs in ~60s and Flow finishes ~18s on a 4‑core box (roughly a 3–4× win), and the example pipeline (stream → flow → partition → reduce) works.
AI-written summary · about
chapters (5)
related talks
40mTurn Hours into Seconds- Concurrent Event Processing
41mA practical guide to GenStage with a real life use case
25mLeveraging GenStage to Implement Your Own Event Bus
45mKEYNOTE: Announcing Broadway
50mOperation Flow
46mMulti-Priority Graph-Based Parallel Execution Using Taskflow
01h 01mTaskflow: A Parallel and Heterogeneous Task Programming System Using Modern C++
37mRow, row, row your boat gently with Flow