Model Serving for Deep Learning
summary
Open source Model Serving for Apache MXNet provides a concrete deploy path: pip install MXNet model server installs a CLI that downloads a 5 MB SqueezeNet model, serves it on localhost and returns an 85% “Egyptian cat” prediction in the demo. Production use relies on the MMS docker image with nginx, GPU/process auto‑forking and orchestration recommendations. Models package into a model archive (params, signature, optional custom code). Built metrics, CloudWatch reporting, and ongoing work on adaptive batching and latency optimization show it actually works at scale.
AI-written summary · about
chapters (4)
related talks
01h 16mDeep Dive into Deep Learning with MXNet
27mModel Serving at the Edge Made Easier
42mAlex Smola, Professor in the Machine Learning Department, Carnegie Mellon University
33mHigh Performance Deep Learning on Edge Devices With
20mApache MXNet, a highly memory efficient deep learning framework
23mBest Practices for Building Production-Grade Deep Learning Systems
36mWhat's New, ModelMesh? Model Serving at Scale
29mAccelerate NumPy Data Science Workloads and Deep Learning Applications