Dask at Global Scale with
summary
Claim: Coiled offers managed Dask so users get a real cluster from two commands — pip install coiled and a couple Python calls — with software environments, credentials, cost controls and network security handled for them. Demo runs on AWS (Azure soon), loads a 20–30 GB NYC taxi dataset across 10 machines, shows ~42 cores in use, enforces a 200 core limit, supports GPUs and XGBoost and has a ~1 minute spin up time while remaining free to try.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:52Dask for parallel computing
- 8:12Dask architecture and growth
- 13:07IT challenges with Dask
- 17:40Coiled introduction
- 20:09Coiled simplifies Dask
- 25:47Closing and future
- 26:49Q&A on Dask challenges
related talks
01h 26mDask tutorial
01h 23mData Science at Scale with Dask
01h 03mDask: Next Steps in Parallel Python
30mDask ❤️ Xarray: Geoscience at Massive Scale
25mDeploying Python at Scale with Dask
44mDask for ad hoc distributed computing
01h 22mUsing Dask for Parallel Computing in Python
01h 17mScaling PyData with Dask