Model serving: which framework to choose?
summary
Overview of frameworks for serving machine learning models in enterprise settings, comparing several model-serving frameworks by implementing the same task in each. The talk evaluates differences between these frameworks to help decide which one to choose for production model serving.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 3:17What is Serving Engine
- 6:43Framework comparison
- 11:23Supported model formats
- 18:29Model pipelines
- 21:19Kubernetes deployment
- 23:27API interaction
- 26:15Monitoring and metrics
- 27:44Performance optimizations
- 31:42Development complexity
- 33:07Choosing a framework
- 36:40Q&A
related talks
48mDeploying Machine Learning Models in Practice
31mServing Machine Learning Models at Scale Using KServe
26mPatterns of ML Models in Production
25mRay Serve: Patterns of ML Models in Production
38mServerless Architecture for Machine Learning
40mScalable machine-learned model serving
40mServing Machine Learning Models at Scale Using KServing
34mServing ML Models at Scale with Seldon and Kubeflow