Hardware & Co-Design
summary
MTIA V2 is a TSMC 5nm, 90 W chip running at 1.35 GHz with 177 TFLOPS (FP16/bf16) and a 2.35 billion gate design, built around a very large on‑chip SRAM and off‑chip LPDDR; custom dot‑product engines and embedding offloads plus int8/int4 and fb16 support drive efficiency. Software is PyTorch‑native + Triton with torch Dynamo/Inductor, kernel autotuning and a model compiler; job launch latency drops to ~20% of V1 and table‑embedding ops run 2–3x faster. Low‑complexity ranking beats baseline rack perf and perf/W, high‑complexity models need more kernel/SRAM blocking work.
AI-written summary · about
chapters (4)
- 0:00Model Co-design for MTIA
- 6:59MTIA Next Generation Accelerator
- 13:46Silicon Software
- 19:18Closing Thoughts
related talks
24mIndustrial Strength High Performance RISC V Processors For Energy Efficient Computing
21mMTIA: Meta's First Generation of AI Accelerators
25mRace to Superintelligence: Building Low-Friction AI Infrastructure for a Heterogen...
12mAI-powered infrastructure:
18mCo-Designing Communication for AI Accelerators
11mAccelerated Software for a Post-Moore World
20mLow Power AI Edge Hardware
24mPortable High‑Performance LLM Serving: A Triton Backend for...