Speeding ML model inference without unnecessary cost
summary
Describes how DomKlik uses neural networks for voice and text bots and examines when moving models to GPUs is not the best option; discusses practical trade-offs and alternatives for accelerating ML inference without unnecessary cost.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 2:55Current approach problems
- 5:49OpenVINO overview
- 7:43Model architecture breakdown
- 10:15Postprocessing optimization
- 12:01ONNX model fusion
- 13:52Testing and results
- 17:25GPU comparison
- 19:34OpenVINO usability
- 20:17Model quantization
- 22:45Quantization training
- 23:49Model pruning
- 24:56C++ postprocessing
- 26:00OpenVINO alternatives
- 26:44Summary
- 28:49Q&A
related talks
24mCost and Performance Optimization of LLM Inferencing.
32mTaming the Machine: Basics of ML Models Training and Inference Optimization
49mMobiconf 2018: Vladimir Alekseichenko, DataWorkshop
41mEscaping the Cloud: High-Performance AI in your Browser
34mOptimizing AI Inference with ML Compilers & Hardware
47mMachine Learning on Android Demystified
12mPreview: ML Serving
30mGo, ONNX and AI in Production