Extracting real-time data at the edge with SageMaker & NVIDIA GPUs

on YouTube ↗45m

" summary

Pipeline shows a people‑counting system that compiles a YOLOv3 pupil detector with SageMaker Neo, optimizes with TensorRT, and deploys a Neo/TensorRT artifact to an NVIDIA Jetson Nano (JetPack 4.6) via AWS IoT Greengrass. DeepStream runs the NVInfer plug‑in, resizes live 720p RTSP to ~224×224, and passes tensors either via shared library or gRPC to the Edge Manager. Demo runs reliably, counts entries/exits correctly, and supports over‑the‑air model updates and fleet rollbacks.

AI-written summary · about

" chapters (29)

1. 0:00Intro
2. 2:15Al inference drives the modern application
3. 3:06The challenge with Al at the edge
4. 4:39NVIDIA Jetson
5. 5:22The Jetson Family
6. 6:56The secret is continuous software advancements
7. 8:07Jetson software
8. 10:33Inference is complex
9. 11:04NVIDIA TensorRT
10. 11:35Inference optimization workflow for TensorRT
11. 12:18TensorRT integrated with PyTorch and TensorFlow
12. 12:38Framework integrations
13. 13:21Al inference deployment options
14. 14:21Layer & Tensor fusion
15. 15:26Kernel auto tuning
16. 16:24CV is more than Al inference
17. 17:10NVIDIA end-to-end SDK
18. 17:51Pipeline efficiency with O memory copies
19. 18:43DeepStream SDK
20. 20:36DeepStream application
21. 21:56AWS IoT Greengrass edge capabilities
22. 25:07Edge runtime - Build device software quickly
23. 27:37How SageMaker Edge Manager Works
24. 28:28Edge ML basics: Amazon SageMaker Neo
25. 29:39Edge ML advanced: NVIDIA DeepStream integration
26. 35:44Getting started with Amazon SageMaker Edge Manager
27. 36:01Amazon Lookout for Vision
28. 36:48Quality management with Lookout for Vision
29. 41:00How does AWS Panorama work?

" related talks

• 53mVideo Analytics for mere mortals with NVIDIA DeepStream SDKYousry Mohamed · NDC Melbourne 2022
• 28mMachine Learning on the EdgeFrank Kelly · PyCon UK 2019
• 18mDeploying ML solutions with low latency in PythonAditya Lohia · Conf42 Machine Learning 2021
• 47mBest Practices for Real-time Intelligent Video AnalyticsEkaterina Sirazitdinova · GOTO Copenhagen 2021
• 55mLarge Scale Distributed AI on the JVMMo Haghighi · Devnexus 2020
• 20mAccelerating AI pipelines w/ NVIDIA on AWS: An autonomous driving caseNorm Marks · AWS re:Invent 2019
• 32mMLOps for edge devices with Amazon SageMaker Edge ManagerVin Sharma · AWS re:Invent 2020
• 54mDeep learning on AWS with NVIDIA: From training to deploymentJiahong Liu, Edwin Weill · AWS re:Invent 2022