Scaling Interactive Data Science with Modin and Ray
summary
Modin runs as a drop-in replacement for pandas with native Ray and Dask execution and a plugin to hand work to external engines like OmniSci via Apache Calcite. It implements about 93–94% pandas API (vs ~45% for Dask and ~58% for Koalas) and proves the design on a 100M row NYC taxi benchmark where Modin shows roughly 100x advantage over Koalas and 5.6x speedup on true medians versus pandas. The system is modular, falls back to pandas when needed, and is production-ready.
AI-written summary · about
chapters (8)
- 0:00Introduction and speaker background
- 2:28Human scalability in data science
- 6:19Formalizing the data frame model
- 11:17Modin's pluggable API design
- 13:25Unified workflow across environments
- 16:56Performance comparison with other systems
- 19:24Modularity and future integrations
- 20:05Conclusion and open source call
related talks
21mScaling Your Pandas Analytics With Modin
29mLearning to Scale Data Science, Machine Learning, and Pandas with Ray and Modin
24mData Scientists are More Valuable than Hardware: The Design of Modin
46mScaling Data Science with Modin
39mAccelerated Data Science Libraries
01h 22mHigh-Performance Data Science at Scale with RAPIDS, Dask, and GPUs
30m"Better dataframes"
39mAccelerating Data Science with RAPIDS