Convex Hull Approximation for Activation Functions
summary
WraAct constructs tight polyhedral over-approximations of activation convex hulls by introducing linear constraints that smooth activations into double-linear-piece (DLP) functions and then over-approximating those efficiently. Claims: average 400X faster and 150X more precise than SBLM+PDDM while using ~50% fewer constraints, handles up to 8 input dimensions within 10 seconds, boosts single-neuron verification on 100 benchmarks from under 10 to over 40 verified samples, and adds up to 20 verified cases over PRIMA; verifies a 22k-neuron ResNet in about one minute per sample.
AI-written summary · about
chapters (10)
- 0:00Convex hull approximation intro
- 1:30Neural network verification
- 3:03Mainstream verification methods
- 4:32Multi-neuron constraints
- 6:02Problem formulation
- 7:10Double linear piece functions
- 9:06Efficient DRP approximation
- 10:52Active linear constraints
- 11:13Evaluation results
- 12:22Q&A
related talks
23mPRIMA: General and Precise Neural Network Certification via Scalable Convex Hull
18mReLU Hull Approximation
30mBoosting Multi-Neuron Convex Relaxation for Neural Network Verification
31mAbstract Neural Networks
31mARENA: Enhancing Abstract Refinement for Neural Network Verification
20mSupport Triangle Machine
19mAccelerating the Training of Large Language Models using Efficient Activation...
21mArchitecture-Preserving Provable Repair of Deep Neural Networks