Neural Networks and Accelerator Co-Design
summary
Co-design of network and FPGA gives concrete gains: using Brevitas for quantization-aware training plus the FIN toolchain produces hardware-mapped networks that trade precision for throughput (example: binary ops boost peak compute ~200x to ~66 TOPS and on‑chip weight capacity from ~2M to ~70M). Streamlining folds batch‑norm and bias into thresholds and XNOR+popcount logic so weights stay integer, and a streaming dataflow design hits examples like 300M inferences/sec at ~18 ns latency — demonstrated on real FPGA builds and open‑source repos.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 1:30Motivation
- 3:47About Xilinx
- 4:42What is an FPGA
- 5:52What are DSP blocks
- 9:32About the group
- 11:28The problem
- 16:25Finn
- 22:10Quantize Neural Networks
- 22:25Why Quantize Neural Networks
- 23:45Custom Integer Operations
- 24:28Power and Performance
- 26:22Configurable Logic Blocks
- 28:10Accuracy
- 34:38Quantization
- 38:35Quantization Techniques
- 43:19Straight Through Estimator
- 48:36Brevitas
- 53:18MobileNet V2
related talks
34mBuilding FPGA-based Machine Learning Accelerators in Python
18mCo-Designing Communication for AI Accelerators
01h 11mMachine Learning Acceleration in an Embedded World
30mAccelerating Neural Networks for Vision Systems via FPGAs
09mAccelerating compute with software defined hardware (FPGA's)
13mFENIX: Enabling In-Network DNN Inference with FPGA-Enhanced Programmable Switches
18mFLASH: Towards a High-performance Hardware Acceleration Architecture for Cross-silo...
13mHardware/Software Co-Programmable Framework for Computational SSDs to Accelerate Deep...