Optimizing workloads with Transformers and Diffusers
summary
Shows concrete speedups and exact steps: using Hugging Face Transformers pipeline yields summarization in 42s, switching to Optimum cuts it to 27s; converting a classifier to ONNX Runtime moves eval time from 254s to 182s, applying ORT Optimizer to 158s, then quantizing to 66s while accuracy stays ~0.57. For vision, running Stable Diffusion 1.5 with diffusers on CPU (25 steps) is ~14s, Optimum/OpenV reduces it to ~10s, and GPUs go from ~4s (A10) down to ~1.3s (A100) using lower precision and static shapes.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:05Transformers architecture overview
- 12:20Live demo: Transformers optimization
- 27:13Diffusers introduction
- 34:05Stable Diffusion inference
- 43:36Live demo: CPU optimization
- 53:32Optimizing Stable Diffusion
- 1:03:46Live demo of stable diffusion on different GPUs
- 1:14:54Q&A
related talks
23mThe Race to the Bottom: Low Latency in the age of the Transformer
18mDeploying ML solutions with low latency in Python
25mtorch.compile and Diffusers: A Hands-On Guide to Peak Performance
23mOptimize your network inference time with OpenVINO
36mHigh-Performance Distributed Tensorflow
36mAccelerating High-Performance Machine Learning at Scale i...
42mHigh Performance Distributed Tensorflow
25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and Quality