Understanding GPUs
summary
Shows concrete tradeoffs and numbers: using torch and tensorflow maps deep nets to GPUs so forward/backward passes go from weeks to hours (one example cuts a week to four hours), multilayer perceptrons get a 20–30x speedup, convolutions scale well, GMMs hit ~25x, CUDA tree gives ~5–6x, while RNNs and many sequential or memory‑bound algorithms barely improve. Notes practical limits like 12 GB GPU RAM vs 128 GB system RAM, costly multi‑GPU engineering, and prefers batching or local GPUs over naïve porting.
AI-written summary · about
chapters (22)
- 0:00Intro
- 1:25What are GPUs Good For? Cryptocurrency
- 2:44What is CUDA?
- 3:59Theoretical CUDA
- 4:21Adding Two Arrays
- 4:57Summing an Array
- 5:33Matrix Multiply
- 6:52Some Terminology
- 7:12A Grid of Blocks of Threads
- 7:29Real World CUDA Threads
- 8:48Summing Two Arrays
- 9:28Real World CUDA Memory
- 15:12CUDA in Unexpected Places
- 16:16Classification Trees
- 18:19CUDA Trees
- 21:52Multi GPU Setups
- 22:53Sequential Algorithm
- 24:08Algorithm Changes
- 25:25Technical Debt
- 26:16No Streaming
- 27:05CPUS
- 29:06Opinions - Multi GPU
related talks
38mHow to run Neural Nets on GPUs'
01h 12m30 Golden Rules of Deep Learning Performance
33mWhen GPUs Make Python Slower: Patterns and Pitfalls
41mLuxuries, necessities, and the challenges that remain
36mWhy Are GPUs (Not) Fast - A Trip Through the Driver Stack
18mTC-GNN: Bridging Sparse GNN Computation and Dense Tensor Cores on GPUs
44mArchitecture Effects When Interacting with GPU Technologies
13mFrom Silicon to Scale: Training at Scale