Introducing ExecuTorch from PyTorch Edge: On-Device AI...
summary
ExecuTorch launches as a PyTorch Edge on‑device stack that exports models with torch.export, compiles ahead‑of‑time, and delegates subgraphs to Arm Apple and Qualcomm backends. Runtime is a 40 KB embedded C++ engine (previous builds were ~1.5 MB), supports about 200 core A10 operators, and exposes Python APIs for quantization, memory planning and profiling. It ships on Ray‑Ban and Meta smart glasses, targets Quest 3, and can run Llama 2 with quantization, efficient KV cache and dynamic‑shape compilation.
AI-written summary · about
chapters (16)
- 0:00Introduction
- 0:35History of PyTorch
- 1:46OnDevice AI
- 3:24Design Principles
- 4:28Introducing ExecuTorch
- 5:20Why ExecuTorch
- 6:54Tradeoffs
- 9:11First Benefit
- 10:27Second Benefit
- 13:55Customizability
- 14:21Delegation
- 15:33Design
- 16:35Benefits
- 17:06Roadmap
- 17:43ExecuTorch Llama 2
- 19:12Quantisation
related talks
23mBringing ExecuTorch To the Next Frontiers of Edge AI
33mExecutorch: Enabling and Accelerating PyTorch Models on Edge Devices
25mExecuTorch 1.0: General Availability Status for Mobile and Embedded...
10mExecuTorch on Microcontrollers: Deploying PyTorch To...
08mUnlocking AI at Scale for PyTorch Developers
25mLeveraging PyTorch for Generative AI in Distributed Edge Clouds
13mRunning ExecuTorch Applications With Silicon Accelera...
25mOn-Device LLM Inference on Android With ExecuTorch and Qualcomm QNN