Overcoming data infrastructure limitations with a new paradigm for ML
summary
Kaskada describes a system that processes events chronologically and partitions by entity (user, age, show) to compute feature streams as of each event, enabling point-in-time lookups and composable time shifts so targets and features align without backfills. It snapshots accumulator state for incremental production serving, trades extra shuffles for temporal correctness, and claims practical performance (roughly 10–20s for 10M examples). Guarantees forbid shifting values backward, preventing temporal leakage via native time travel.
AI-written summary · about
chapters (9)
- 0:00Introduction to talk
- 1:25Current ML tools limitations
- 7:21Chronological processing approach
- 11:32Distributing computation with entities
- 13:59Point-in-time lookups explained
- 21:00Native time travel concept
- 26:57Productionizing queries incrementally
- 32:12Recap and benefits summary
- 35:03Q&A session
related talks
17mCreating & operating ML models from event-based data using feature stor...
41mUsing Kaskada to Do Real Time AI
21mTime for quality ML from event-based data
33mUsing Kaskada to do Real Time Machine Learning- 41mUsing Kaskada to Do Real Time AI
08mReal-Time Data Systems: Speed, Scale, and Resilience
47mExpert Panel: Current & Future State of the Art in Real-Time Data Infrastructure
29mThe need for speed: Data streaming architectures