Video Analytics for Football games

on YouTube ↗51m

" summary

Pipeline ingests HLS via FFmpeg, publishes frames to Pub/Sub, and uses Apache Beam Dataflow with Python SDK to window at 10 fps (3s windows, trigger after 30 frames) for per-frame detection and sequence models. Models run on TF‑Serving on Kubernetes with GPUs; outputs go to Bigtable for a web player. Ball model needs about 700 extra ball labels and start-of-game labeling bootstraps to ~2,000 events. System targets a 10–20 second delay and wrestles with GPU autoscaling and watermark workarounds.

AI-written summary · about

" chapters (28)

1. 0:00Intro
2. 2:29PLAYER AND BALL CAN BE DETECTED PER FRAME
3. 3:42EVENT DETECTION REQUIRES SEQUENCE OF FRAMES
4. 4:51PROJECT CONTEXT
5. 5:33THE PROBLEM LANDSCAPE
6. 7:39TWO SOLUTION PARTS
7. 9:06THE DATA FACTS
8. 9:50LEVERAGE THE MODEL TO SPEED UP THE LABELING
9. 10:55THE MODELS
10. 11:45PLAYER DETECTION: FIELD TRANSFORM
11. 13:36A WALK IN FEATURE SPACE
12. 14:16SUBTRACT BACKGROUND TO REMOVE THE NOISE
13. 17:13COORDINATES AS UNLOCKED DOWNSTREAM FEATURE
14. 19:14START OF GAME MODEL BEATS THE OTHER GOAL MODELS (FOR NOW)
15. 20:34SOLUTION ARCHITECTURE
16. 22:13ABOUT APACHE BEAM
17. 23:09THE SOLUTION LANDSCAPE
18. 24:05FROM HLS TO JPEG
19. 28:37FULLY LEVERAGE MANAGED SERVICES
20. 30:51LEVERAGE THE BEAM MODEL FOR PROCESSING
21. 32:49WHERE THE DATA CRUNCHING HAPPENS
22. 36:55PIPELINE DEEP DIVE
23. 39:30LEVERAGE THE INTERNAL LOAD BALANCER OF GKE TO GET PREDICTIONS
24. 40:24DEWARPING THE BOUNDING BOXES TO GET COORDINATES
25. 40:59TEAM DETECTION WITHOUT BACKGROUND SUBTRACTION
26. 41:53LEVERAGE THE BEAM MODEL TO WINDOW THE DATA
27. 43:16RESPECT THE BEAM MODEL TO GET DESIRED PARALLELIZATION
28. 48:20TEST IN STREAM MODE

" related talks

• 20mAI-Driven Real-Time Ball Tracking for Live Sports StreamingThomas Davies · Demuxed 2025
• 35mFootball video analysis using Deep LearningJacek Komorowski · PyData Warsaw 2019
• 42mDeep Learning for Automatic Basketball Video ProductionJulian Quiroga · CVPR 2020
• 36mVideo analytics at scale: Deep Learning, Computer Vision, ML & SparkClaudiu Barbura · Global Artificial Intelligence Virtual Conference Sep 2020
• 36m"Lights, camera, AI!" - Automated sports videographyZack Akil · PyData London 2017
• 27mAnalysis of sports scenes using computer vision modelsMarina Ivasic Kos · DSC Europe 2022
• 53mVideo Analytics for mere mortals with NVIDIA DeepStream SDKYousry Mohamed · NDC Melbourne 2022
• 32mCutting Edge Football Analytics in PythonJoris Bekkers · PyData London 2025