Reproducible machine learning and science with python
summary
Claims that machine learning experiments become reproducible by recording the actual artifacts: openml-python is used to publish exact datasets models runs benchmarks (dataset metadata, task definitions, flows, run evaluations), plus a web UI and APIs (Python R Java C++) and scikit‑learn integration. Surprising scale: about 150k users 20k datasets 10M runs and ~5k active datasets. Built items: benchmark suites, queryable task lists, and export to parquet; limits: no model weight uploads, PyTorch support is alpha and hardware/seed nondeterminism remains.
AI-written summary · about
chapters (26)
- 0:00Start
- 2:43Reproducibility in Machine Learning
- 3:55Models
- 4:54Openml
- 5:51Simple Machine Machine Learning Workflow
- 6:46Machine Learning Libraries
- 7:52What Is Metadata
- 9:16What Is a Task
- 10:17Auto Annotation
- 10:50Python Api
- 11:26Frictionless Machine Learning
- 12:28Get Data Set out of Psychic Learning
- 13:12Reproducibility by Openml
- 15:39About Openml
- 16:19All the Projects under Open Ml Umbrella
- 18:06Openml Cc18 Benchmark
- 22:08Manipulate the Statistics of the Models inside the Run
- 24:34Openml Is More for Research Groups or Also for Enterprise Project and What if a Pipeline Should Be Reproducible within the Team but Not Publicly
- 29:57Data Set from Openml
- 37:22Task Tutorial
- 42:10Openml100
- 47:33Cost Matrix
- 50:09Tic-Tac-Toe Classification
- 53:42Create the Supervised Classification Task on Dataset 128
- 1:04:35Benchmark Suites
- 1:04:45Benchmark Suite
related talks
30mReproducible Machine Learning
30mReproducible Machine Learning at Scale
29mReproducibility in ML Systems: A Netflix Original
28mTrusted AI – Building Reproducible, Unbiased and...
20mBetter Care and Feeding of Machine Learning Models
30mRealizing End to End Reproducible Machine Learning on Kubernetes
32mReproducibility and versioning of ML systems
33mCompliant Data Management and Machine Learning on Kubernetes