Building Abstractions at the Hardware-software Boundary
summary
Claim: a deterministic spatial chip with software-controlled memory and lockstep execution lets the compiler schedule every vector operation across time and space, model performance pre-silicon, and avoid runtime unpredictability. Implementation details: 320-element SIMD vectors, ~220 MB on-chip, 80 TB/s bandwidth, <3% area for dispatch, simple 1D stream registers. Tooling: Groq View visualization, TSP compiler and FPGA-style chip model enable cycle-accurate scheduling. Result: automated mapping replaces ad hoc kernel libraries and yields narrow tail latencies (BERT base shows state-of-the-art latency and tight 99th-percentile behavior).
AI-written summary · about
chapters (20)
- 0:00Start
- 3:49The Instruction Set Architecture
- 11:37Intel Titanium
- 12:53Breakthrough for Deep Neural Networks
- 14:12The Hardware Lottery
- 18:07Building Blocks
- 21:12Unpredictable Memory Latency
- 22:05Thread Synchronization
- 23:19The Complexity of the Interconnect
- 25:47Software Controlled Memory
- 26:11Memory
- 29:16Importance of Interconnect
- 43:05Grok Compiler Flow
- 44:33Templated Polymorphism
- 49:13Abstraction Model of the Architecture
- 52:59What Is the Programming Model
- 53:25Conditional Branches
- 54:35Programming Model
- 1:00:14Examples of Cloud Language
- 1:01:06Grok Api
related talks
01h 04mDesign for Performance
01h 37mAn Overview of Program Optimization Techniques
57mAdventures in SIMD-Thinking (part 1 of 2)
01h 00mSYCL: Open Your Code to More Hardware Choices
55mAbusing Your Memory Model for Fun and Profit
54mCost of Abstractions in Embedded Systems
58m“No Sane Compiler Would Optimize Atomics"
01h 06m“Improving Performance Through Compiler Switches..."