Store and manage data effortlessly with HDF5
summary
Claims that HDF5 gives accessible compressed organized storage with real code and numbers: uses h5py and an IPython notebook to show a processing pipeline that replaced MATLAB, handles MEG time series with ~248 sensors per scan and thousands of subjects, and demonstrates slicing, chunking, compression, h5dump and HDFView/HDFCompass and MPI parallel I/O. Examples show files with groups datasets and attributes, an implemented pipeline on real brain recordings, and that workflow scales and remains readable and shareable.
AI-written summary · about
chapters (7)
- 0:00Introduction
- 7:21What is HDF5
- 16:14HDF5 structure
- 20:09Python HDF5 files
- 24:26Real-world HDF5 example
- 25:08Key benefits
- 26:35Q&A
related talks
01h 20mAdvanced Data Storage
31mh5pydantic, from a Synchrotron through Python to HDF5
45mInteractive Analysis of (Large) Financial Data Sets
01h 01mParallel Python Analyzing Large Data Sets
52mZarr vs. HDF5
44mUsing Dask for Parallel Computing in Python
31mAccelerating Single-cell Bioinformatics with N-dimensional Arrays
25mpypet: A Python Toolkit for Simulations and Numerical Experiments