Faith: An Efficient Framework for Transformer Verification on GPUs
summary
Faith implements a GPU framework that fuses bound-centric operations to cut redundant global memory accesses, loads weight once into shared memory, and accumulates output bounds in registers, using warp shuffles and shared/register collaboration to speed reductions. It combines semantic-aware computation graph transformation, a verification-specialized kernel crafter that maps three computation patterns to threads, and expert-guided autotuning via hardware meta files. End-to-end results show 2.1–3.4× (2.6× average) speedup over prior frameworks, ~2.5× vs PyTorch and up to 5.1× on matmul.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:20Motivation: Transformer Applications
- 0:53Motivation: Security Concerns ⚫ Synonym Substitution Attack
- 1:40Motivation: Transformer Verification
- 2:23Challenges: Unique Computation Pattern
- 2:55Computation Patterns
- 5:28Overview
- 6:14Sematic-aware Computation Graph Transformation
- 8:31Diversity across Verification Designs
- 9:45Verification Pattern Categorization
- 10:57Workload Adaptive Reduction
- 12:17Sharing-oriented Workload Scheduling
- 13:51Expert-guided Autotuning Optimization
- 15:38Evaluation: End-to-End Benefits
- 16:25Evaluation: Per-layer Benefits • Matrix Multiplication
related talks
15mSAVE: Software-Implemented Fault Tolerance for Model Inference against GPU Memory..
29mEnabling Lightweight, High-Performance FSDP With NVIDIA GPU
16mGalvatron: An Automatic Distributed Training System for Efficient Large...
01h 18mOptimizing LLM Training and Inference Performance on GPUs
25mProof transfer for fast certification of multiple approximate neural networks
26mDeepSpeed: Efficient Training Scalability for Deep Learning
21mKTransformers: Unleashing the Full Potential of CPU/GPU Hybrid Inference for M...
08mHacks to Make LLM Training Faster