Real Time Machine Learning
summary
Claim: retraining frequently preserves and even improves models — weekly retraining raises accuracy ~11% and F1 ~20% on a flight delays example. A core set tree built once takes 8s vs 335s to train, lets adding 200k new rows cost 4s via partial build, and keeps retrain time near 28s. Hyperparameter search drops from 154h to 5h and accuracy moves from 0.886 to 0.94 using the data Heroes corset approach.
AI-written summary · about
chapters (13)
- 0:00Real-time ML intro
- 1:07Model degradation over time
- 3:14Frequent retraining challenges
- 3:58Automated real-time ML framework
- 5:44Avoiding compute costs with corset trees
- 8:13How corset trees work
- 10:39Updating production models
- 11:40Retraining efficiency example
- 13:48Corsets and sensitivity values
- 16:40Case studies: accuracy and cost
- 18:45Hyperparameter tuning with corsets
- 19:47Getting started with Data Heroes
- 21:53Q&A
related talks
28mWhy we do ML model retraining wrong, and how to do better
31mFine-tuning LLMs: A Cost-Benefit Analysis for Businesses
29mAutomated Model Update & Evaluation
25mMachine Learning is Going Real-Time
33mBuild Better AI with First-Party Data and Fine-Tuning
40mProductionizing Machine-Learning Services: Lessons from Google SRE
31mFoundational Time Series Models in Practice: The Future of Forecasting, or Just Hype?
31mAdvanced machine learning time series....