Open Source meets Enterprise: The right way.
summary
Shows concrete changes: Dask QN reduces memory use ~80%, with a live demo where the old scheduler spills to disk and the new one runs about 6–7x faster on the shown workload, a community-driven scheduler change actually landed in Dask. Adds measurable improvements from ecosystem work—Arrow strings cut worker needs from ~45 to ~15 for a taxi dataset—and production observability: continuous large-scale benchmarks run daily by Coiled so regressions and performance gains are visible and reproducible.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 2:51Open source journey with Dask
- 5:41What is Dask
- 9:18Dask governance
- 12:25Performance improvements
- 14:30Demo: TaskQM memory optimization
- 17:38Nvidia’s GPU contributions
- 19:00Company collaboration in Dask
- 20:28Benchmarking at scale
- 24:01Why multi-company collaboration works
- 25:48Conclusion & Q&A
related talks
29mOpen Source Powers The Modern Data Stack
16mKeynote: Accelerating AI/ML Development with Open Source Software
28mHow Open Source Enables Innovation A Case Study with dbt and Lightdash
32mOpen Source: The Engine of Innovation in the Digital Age
45mState of the Duck
29mScaling Grid Security With Open Source
56mWelcome & State of Open Source Databases
01h 39mOSS Keynotes - Day 2 Morning Sessions