Big Data Oceanography
summary
Claims that oceanography workflows scale from single workstations to cluster-backed processing using xarray and dask, handling tens to hundreds of terabytes of model output and autonomous-float data stored as netcdf. Shows a reproducible pipeline idea packaged as the Cosima cookbook that indexes many small files, lazy-chunks and parallelizes zonal means and eddy kinetic energy calculations, demonstrates Docker and notebook-based reproducibility and reports practical feasibility on modest clusters.
AI-written summary · about
chapters (6)
- 0:00Physical oceanography intro
- 3:22Lab experiments workflow
- 10:27Numerical modeling challenges
- 17:30Reproducibility and open tools
- 24:07Scalable Python analysis
- 35:52Why data pipelines excite me
related talks
30mDask ❤️ Xarray: Geoscience at Massive Scale
30mGeoscience at Massive Scale
22mKubernetes Meets Climate Data: Designing Analysis-Ready Dat...
32mDemocratizing Distributed Computing with Dask and JupyterHub
19mA world of data
19mMaking Waves with R, Python, and Quarto
01h 24mData Science at Scale with Dask
29mScaling Data Workloads with Cluster Managers