Disco: Parallelizing Video Processing and Machine Learning with Disco Serverless Computing
summary
Disco claims to let developers run large-scale video and ML workloads with one CLI line and unchanged Python multiprocessing code, by packaging work into a Docker image and letting Disco schedule agents across on‑prem or cloud nodes. It supports multi-CPU and GPU jobs, auto‑scales to thousands of workers, returns results as a ZIP, and reports typical speedups from 2x up to 1,000x (worst‑case network/latency limits apply); not suited for hard real‑time. single line command
AI-written summary · about
chapters (32)
- 0:00Introduction
- 0:30Meet Bray
- 1:34Brays History
- 2:27PhD Thesis
- 3:06Dependencies
- 3:53Industry Problem
- 4:07Mission
- 4:42Lecture
- 5:24Demo
- 8:05Programming Side
- 8:16Multiprocessing
- 9:39How it works
- 10:00Scaleable
- 10:23Architecture
- 11:45Use Case
- 12:49The Problem
- 13:18The Solution
- 13:56The Business Perspective
- 14:59The Big Problem
- 15:21One Solution
- 16:06Same Top Image
- 17:41Example Code
- 18:23Business Decision
- 18:58Single Line Comment
- 20:12Video Processing
- 21:00Customers
- 21:33PR
- 22:05Summary
- 23:34Theres no perfect solution
- 24:37Dont run infinite loop
- 26:09Innovation Center
- 26:22QA
related talks
41mApplying serverless architecture pattern to distributed data processing
46mServerless for Data Scientists
21mFrom Laptop to Lambda: Outsourcing Everyday Jobs to Thousands of Transient Functional Containers
16mBurst Computing: Quick, Sudden, Massively Parallel Processing on Serverless...
51mDancing with Serverless
41mUnlocking Faster & Efficient Data Processing w/ Serverless
31mBirds of a Feather: Do You Really Need on Premises Serverless ?
45mAutonomous Multi-Cloud Serverless Deployment and Optimized Management