Neural Architecture Search using Property Guided Synthesis
summary
αNAS performs evolutionary NAS by mutating abstract program properties (depth, shape, mixing) instead of raw graphs, concretizing candidates via a new progressive synthesis procedure over an 1800-op DSL. It finds ResNet-34 variants on CIFAR-10 with equal accuracy and 96% fewer parameters, and ImageNet models that beat ViT (≈30% fewer FLOPS/params), ResNet-50 (23%/14%), and EfficientNet (≈7%/7%) within 400 generations; synthesis costs are minute-scale, training costs hours.
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:14Architectural innovations are a key driver of progress in ML
- 1:01(Evolutionary) NAS: System overview
- 2:00(Evolutionary) NAS: Search Space
- 3:18(Evolutionary) NAS: Block + subgraph selection
- 3:38(Evolutionary) NAS: Block mutation
- 3:52(Evolutionary) NAS: Architecture mutation
- 4:34Prior approaches: small, random mutations
- 7:41Depth + Shape properties
- 8:27Mixing property
- 13:41Efficient concretization via progressive synthesis
- 17:00Why does this work?
- 20:44Case study 1: VIT-S/16 on ImageNet-1k, Model 1
- 22:38Case study 2: EfficientNet-B0 on ImageNet-1k
- 24:12Effect of abstractions on search
- 24:59Enumerative vs. Progressive Synthesis
- 25:52Conclusion
related talks
38mAutoML for Efficient Neural Architecture Design
17mDrs.NAS: Ultra-Efficient Neural Architecture Search for Recommendation Systems
18mActive Learning for Neurosymbolic Program Synthesis
11mAGNES: Abstraction-guided Framework for Deep Neural Networks Security
38mCorinna Cortes, Head of Research, Google, NYC 2017
23mEnhancing Neural Network Robustness via Synthesis of Repair Programs
01h 10mNeurosymbolic Program Synthesis: Bridging Perception and Reasoning in Real-World(…)
28mVisual Search