Machine learning in production
summary
Claims deployment of over 20 models with 14 online inference endpoints (some multimodel) serving ~30 RPS P99, built on AWS SageMaker with Python project templates, CI/CD and canary releases. Uses Tecton feature store for versioned online/offline features, monitors data and concept drift (PSI, KS tests), enforces schema/contract verification (OpenAPI + Pact + schema fuzzing) and runs backtesting on anonymized historical data. Notes custom work for decimal-as-string fields and ongoing cost and lineage challenges.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:47ML in production history
- 3:55Real-time intelligence at Marshmallow
- 5:45Online inference with SageMaker
- 7:28Data pipeline overview
- 8:57Data quality importance
- 11:02Feature stores for data quality
- 14:13Monitoring model drift
- 16:42Verifying models in production
- 18:29Contract testing for ML
- 21:01Schema verification challenges
- 23:32Back testing model changes
- 25:15Future directions for feature stores
- 26:59Q&A
related talks
28mBuilding Developer-Centric ML Inference Platforms
20mIndustrial-Scale Machine Learning
32mAutomating Production Production Level Machine Learning on AWS
26mManaging ML Models @ Scale
40mBuilding highload ML powered service
30mOperationalizing Machine Learning
33mDealing with the versioning of production-ready models
28mScaling Machine Learning Training and Inference