Keynote: Do Repeat Yourself - Designing Open-source Libraries for Modern Machine Learning
Talkat Open Source Summit Europe 2023 · on Thursday, 2023-09-21· at 09:45· Auditorium (Floor 2), Abandoibarra Etorb., 4, 48011 Bilbao, Bizkaia, Spain·
summary
Argues for a purposeful Do Repeat Yourself strategy instead of strict DRY, applied to Hugging Face’s Transformers and Diffusers: each model lives in a single model file policy, many implementations are deliberately copied and kept synchronized by code generators, and a huge test suite enforces a uniform API. Gives concrete metrics: ~750,000 daily pip installs, 100k+ GitHub stars, 2,000 external contributors, ~20k forks, ~400k LOC and ~60,000 tests run daily, with ~40–50 commits to main per week.
AI-written summary · about
chapters (8)
- 0:00DRY principle debate
- 2:04DRY for ML libraries
- 5:33DRY in open-source
- 8:21Readability vs abstraction
- 10:28Single model file policy
- 13:01Code generation for duplication
- 14:47Abandoning DRY drawbacks
- 15:28Q&A
related talks
01h 09mReinventing Machine Learning with Transformers and Hugging Face
51mKeynote: Fault Tolerant Machine Learning Operations
27mPlatforms for valuable AI Products- Iteration, iteration, iteration
33mMachine Learning 2.0 with Hugging Face
59mDevOps for Machine Learning Projects
26mCompleting the machine learning loop
01h 17mMaking MLOps uncool again
25mState of Open Source AI Coding Models