AI acceleration in POWER10
summary
POWER10 adds Matrix Multiply Assist instructions with 8 512-bit accumulators and a 4x4 float32 micro-kernel that produces 16 multiplies per instruction; compilers expose this via built-ins in GCC10/Clang12 and types like vector quad and setxi. Open-source libraries (OpenBLAS 0.3.10+, Eigen 3.4, BLIS sandbox, ONNX Runtime 1.9) integrate MMA kernels and packing changes, and prebuilt PyTorch packages appear on the rocket channel. Measured result: running a standard transformer/inference model on Power9 vs Power10 shows about 5x throughput at FP32 using the MMA-enabled stack.
AI-written summary · about
chapters (7)
- 0:00Introduction to AI acceleration
- 2:30Matrix Multiplication Assist basics
- 8:13Compiler support for MMA
- 13:08Library optimizations for MMA
- 16:23Power10 hardware capabilities
- 20:56Performance results on E1080
- 23:48Q&A and closing
related talks
25mAI Acceleration Capabilities Using MMA(Matrix Multiply...
12mLightning Talk: Accelerating Inference on CPU with Torch.Compile
44mArchitecture Effects When Interacting with GPU Technologies
26mKeynote: Artificial Innovation Accelerates Artificial Intelligence
27mAccelerating Generative AI
21mAhead of the Plasticity Curve: AI, Biology and Technology
12mAI-powered infrastructure:
01h 00mThe Blaze High Performance Math Library