Computers for Super-Human AI
summary
Graphcore develops a chip-based stack (a Colossus microprocessor with ~1,472 parallel cores per chip) and argues the path to brain-scale models uses routing models (sparse activation) so each input touches only a subset of weights. A proposed system packs ~1 petabyte DRAM across ~68 racks (≈2.5 MW, ~$50M) to hold ~100 trillion parameters, enabling human-scale training by trading dense compute for massive high-bandwidth memory and selective parameter access.
AI-written summary · about
chapters (13)
- 0:00Introduction and speaker welcome
- 0:26Purpose of Graphcore hardware
- 1:51Defining ultra intelligence
- 3:17Capacity vs capability machines
- 5:44Human brain parameter scale
- 8:13AI software and hardware today
- 11:44Master learning algorithm limits
- 13:30GPU improvements and limits
- 15:14Silicon scaling challenges
- 17:00Brains vs artificial neural networks
- 19:30Breakthroughs in AI model design
- 21:58Building superhuman AI machines
- 23:23Future of AI and takeaways
related talks
01h 14mTissue vs silicone musings on the future of deep learning hardware and software
43mKeeping Moore's law alive: Neuromorphic computing
51mComputing like the Brain: the Path to Machine Intelligence
58mKeynote: Enabling the AI Revolution: Next-Generation Memory Solutions for AI Computing(…)
13mFrom Silicon to Scale: Training at Scale
21mOrganoid Computing – Can Nature Save AI?
01h 16mRethinking Computing with Neuro-Inspired Learning: Algorithms to Hardware
22mCurrent State of AI