Lightning Talk: Torch-Spyre: Compiling To a Multi-core Dataflow Accelerator...
summary
Torch-Spyre compiles PyTorch models to the IBM Spyre accelerator using Inductor, replacing a significant fraction of proprietary compiler code and improving compilation time and operator coverage without losing inference performance. The hardware is a SoC with 32 compute cores each with compiler-managed scratchpad memory and an interconnect optimized for multicast, and the software uses device-specific tiled tensor layouts and new compiler passes for core-level work division and scratchpad management. Tiling targets 64x64 tiles and symbolic stride-map transforms convert host indices to device coordinates.
AI-written summary · about
chapters (10)
- 0:00Torch-Spyre project
- 0:41IBM zEnterprise accelerator
- 1:22Compute core architecture
- 2:45Matrix multiply with systolic arrays
- 3:26Tile tensors for memory
- 4:08PyTorch device integration
- 6:55Tensor layouts in device code
- 8:17Symbolic index mapping
- 9:00Rebuilding compiler with Inductor
- 10:02Future plans and contributions
related talks
23mBlazing Fast GenAI Inference With Torch.compile
36mDL Compiler Panel Discussion
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
21mThe New Vision of Compiler-Accelerated
07mPyTorch 2.0: TorchInductor
16mLightning Talk: Triton Compiler
30mIntroducing Spark Cyclone Accelerating Spark with...
24mMapping Sparse Applications to Accelerated Computing Systems