Balancing speed and accuracy in model development
summary
Argues that business value comes from choosing speed or accuracy by objective and then applying concrete tools: profile Python preprocessing with Yappi profiler (C-based, use wall time to find hotspots), replace pandas.apply with NumPy or vectorized ops, use grouping to avoid repeated calculations, pick features with SHAP values to prune dimensions, and prefer gradient-boosted trees (XGBoost vs LightGBM) before heavy Transformers. Gives examples: loan aggregator needs results in seconds, medical diagnosis prioritizes accuracy, and recommends measuring trade-offs quantitatively.
AI-written summary · about
chapters (27)
- 0:00intro
- 0:20preamble
- 0:57today's talk
- 1:20the essence of balance: speed vs accuracy
- 1:33factors impacting accuracy and speed
- 2:20the business impact of speed and accuracy
- 3:11real-world examples
- 4:24balancing act: speed, accuracy, and cost
- 6:12strategic importance of the balance
- 6:37how to understand business objectives
- 7:20scenarios for ml-models
- 8:30optimisation strategies
- 8:40training data quality and quantity
- 9:04what is a good dataset?
- 9:18what is a bad dataset?
- 9:50data pre-processing
- 10:40how to find inefficiencies in data pre-processing?
- 11:19yappi
- 14:07most common inefficiencies
- 16:23feature selection
- 17:18shap values for feature selection
- 20:03model selection
- 21:05xgboost
- 22:13lightgbm
- 23:02how to choose the best option
- 23:19a quick recap
- 24:45thank you for your time!
related talks
57mGood Practices for Applied Machine Learning
01h 39mApplying Rigorous Machine Learning Methods in Business Strategy- 57mGood Practices for Applied Machine Learning
37mCreating correct and capable classifiers
32mAn Only One Step Ahead Guide for Machine Learning Projects
26mModel Selection with Python: An Introduction to Hyper Parameter Tuning
39mBeyond Bias: Crafting Fairness in Pythonic Machine Learning Systems
23mDelivering business success with AI