Brevitas Quantization Library
summary
Brevitas is an AMD PyTorch library that composes PTQ stages—rotation transforms, scale/zero choices, and greedy error‑correction—into reproducible pipelines; using Qronos (ICLR 2026) plus Hadamard rotations and AutoRound recovers unusable MXFP4 results (perplexity drop from millions to thousands) and pushes 3‑bit quality past GPTQ. It uses Dynamo tracing to insert rotation ops, integrates with transformers and exports to vLLM/GGUF, and shows end‑to‑end quantization that actually works.
AI-written summary · about
chapters (11)
- 0:00Brevitas intro
- 2:11Quantization for inference cost
- 5:01What is Brevitas
- 7:05Flexibility in quantization
- 9:53Custom rounding functions
- 11:37Post-training quantization stages
- 14:23Algorithm composition for accuracy
- 16:49Kronos error correction
- 20:15End-to-end demo
- 22:37Recap and contributions
- 23:29Q&A
related talks
40mUnlocking Local LLMs with Quantization
19mQuant-LLM: Accelerating the Serving of Large Language Models via FP6-Centric...
48mPractical Quantization in Keras
01h 14mDistributed and concurrent optimization for machine learning
10mQuantization
16mDecDEC: A Systems Approach to Advancing Low-Bit LLM Quantization
25mThe Golden Triangle of Inference Optimization: Balancing Latency, Throughput, and Quality
16mqTPU: Hybrid Tensor Networks for Quantum-Classical Acceleration