AutoML for Efficient Neural Architecture Design
summary
ProxylessNAS automates architecture search directly on target devices by training an over‑parameterized supernet, binarizing architectural choices to keep GPU memory constant, and using a latency lookup table to inject real hardware cost into the objective. It pairs RL-based pruning and AMC compression so searched models hit 74.5% top‑1 / 1.8× latency vs MobileNet‑v2, reduce search cost to about the cost of training one network (~200× reduction over early NAS), and yield hardware‑specific kernels and depths.
AI-written summary · about
chapters (24)
- 0:00Intro
- 0:25History of CNN Architectures
- 2:26Case by case Design - Expensive!
- 3:00From Manual Design to Automatic Design
- 4:35Conventional NAS: Computationally Expensive
- 5:57Conventional NAS: Proxy-Based
- 8:34Our Work: Proxyless, Save GPU Hours by 200x
- 9:43To make NAS 200x more Efficient
- 12:20Save GPU Memory
- 13:16Direct on Target Hardware: Handle non-differentiable Objectives
- 15:21The History of Architectures
- 18:57Results: Proxyless NAS on ImageNet, Mobile Platform
- 21:01Results: Proxyless-NAS on ImageNet, GPU Platform
- 21:45Efficiently Search a Model
- 23:11Detailed Architectures
- 24:19ProxylessNAS for Hardware Specialization
- 25:15Achievements of Design Automation
- 26:39Embrace Open-source
- 30:17Model Pruning
- 30:52AMC: AutoML for Model Compression
- 32:38AMC for Channel Pruning (Acceleration)
- 34:19AMC for Fine-Grained Pruning
- 35:21Multi-precision Inference
- 36:20Accuracy Guaranteed Exploration
related talks
17mDrs.NAS: Ultra-Efficient Neural Architecture Search for Recommendation Systems
30mNeural Architecture Search using Property Guided Synthesis
39mAI/ML on mobile/android with TensorFlow
26mNeuOS: A Latency-Predictable Multi-Dimensional Optimization Framework for DNN-drive
20mArchitecting AI: Efficient Models, Everywhere
40mOptimize Your AI Cloud Infrastructure: A Hardware Perspective
01h 52mNeural Network Accelerator Design
36mApplied AI on the edge