Clean Architecture: How to Structure Your ML Projects to Reduce Technical Debt
summary
Argues that a small, concrete engineering stack reduces ML technical debt: refactor pipelines into a class-based product, run the same code in laptop test and production (inversion of control + dependency injection), and wrap DBs and models with adapter factory strategy patterns. Uses domain classes instead of raw DataFrames, functional tests running ~0.1% of data, and notebook analysis + VS Code production code. Demonstrations show swapping SQL for file sources and changing models without breaking the running product, which speeds iteration.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 0:34Title
- 1:30ML Products
- 3:18Why does technical matter
- 4:30ML lifecycle
- 6:37Technical debt vs technical mess
- 8:27Why refactor
- 9:35What is EOS
- 10:28Decoupling
- 12:24Test Environment
- 12:50Business Problem
- 14:24How does it happen
- 14:48Design patterns
- 17:14Factory Pattern
- 19:36Coherence
- 21:03Workflow
- 23:22Notebooks
- 24:08Benefits
- 25:47Conclusion
- 30:26Unit testing
related talks
44mClean Machine Learning Code: Practical Software Engineering...
28mUnravelling Hidden Technical Debt in ML: A Pythonic Approach to Robust Systems
31m🤑 Paying Off Technical Debt in the Development of Machine Learning Apps
37mBest Practices for Coding in ML/DS - Techniques to Construct your Project
41mCan Data & ML really learn from software engineering?
41mInfrastructure for ML Applications
32mMaking Models Work: How an optimized ML lifecycle can drive better ROI
41mML plumbing