Chain-speed inference for Computer Vision Pipelines
summary
Proposes a concrete pipeline that separates a local storage, a message broker queue and a dedicated machine learning service that must return predictions within the conveyor chain-speed of about 1 to 1.5 seconds. Uses wired streaming over a 10 gigabit switch so eight HD images stream in under 6 ms, queues metadata, and runs batched inference (optimal batch ≈ 8). Compares preprocessing options where a TF dataset costs ~25–30 lines versus two-line methods that are up to six times faster, and uses YOLO + TensorRT for the best speed/accuracy tradeoff.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 1:01Computer Vision
- 3:32Framework
- 4:37Pipeline
- 5:40Data Source
- 6:55Raw Data Transmission
- 8:42Communication Model
- 9:06Machine Learning
- 9:55Garbage in Garbage Out
- 10:39Data Validation
- 11:32Image Transformation
- 11:44Preprocessing
- 13:36Batch vs prediction
- 14:31Model inference
- 15:41Object detection
- 16:47Conclusion
related talks
30mWhen performance matters: running a real-time computer vision pipeline
39mData Pipelining for Real-time ML Models
18mDeploying ML solutions with low latency in Python
22mThe media processing pipelines behind AI
35mWriting Machine Learning Pipelines Against Object Storage
01h 16mOptimizing workloads with Transformers and Diffusers
36mVideo analytics at scale: Deep Learning, Computer Vision, ML & Spark
41mAdding Generative AI to Real Time Streaming Pipelines