Real-Time Revolution: Kickstarting Your Journey in Streaming Data
summary
Bytewax claims a Python-native, stateful stream processor backed by a Rust timely-dataflow engine and demonstrates it by ingesting Wikimedia’s recent-changes SSE, implementing a custom SSE connector, folding events into tumbling windows, and emitting per-user edit counts so you can spot bots. The demo uses a 2 second late tolerance, shows state recovery and replay when the source supports it, and emphasizes portability — runs on Raspberry Pi or Kubernetes and via the Bytewax run module locally.
AI-written summary · about
chapters (8)
- 0:00Real-time data revolution intro
- 2:31Defining streaming data
- 5:43Why streaming needs processors
- 12:26Bwax stream processing
- 16:23Wikipedia event stream analyzer
- 21:11Deserializing and preprocessing
- 25:07Running the data flow
- 26:32Q&A
related talks
30mScaling Embeddings in Real-Time
01h 47mRadanalytics.io streaming applications workshop
45mLive build: How to harness streaming data in real time
44mShaping the Future of Real-time Data Pipeline
39mStreaming doesn’t have to be hard
30mSmart Streams: A Real-time framework for scoring Big and Fast Data
26mBuilding A Python Service Using Open Source Components That Connect, Transform & Deliver Data In RT
01h 02mBig Data Analytics in Near-Real-Time with Apache Kafka Streams