Beyond TorchScript: The Future of C++ Inference in PyTorch
summary
TorchScript is being deprecated in PyTorch 2.9 with a four‑cycle code deletion target around version 24; migration moves models to torch.export (FX graphs with tensor metadata) and the ExecuTorch stack. ExecuTorch reaches GA 1.0 and provides a tiny C++ runtime (~50 kilobytes) for mobile/edge/desktop; ExecuTorch Full for servers is “coming soon” while CUDA/AOT inductor GPU support and server deployments remain experimental. Control flow needs manual rewrite and some legacy features like torchbind are not supported.
AI-written summary · about
chapters (12)
- 0:00Pythonless inference purpose
- 1:04Pythonless inference necessity
- 3:12TorchScript history limits
- 8:14TorchScript IR runtime issues
- 13:33TorchScript deprecation
- 15:44TorchScript sunset plan
- 17:11Introducing ExecuTorch
- 18:38Torch.export frontend
- 20:23ExecuTorch IR stack
- 21:26ExecuTorch runtime targets
- 22:53Next steps future
- 24:30Q&A
related talks
23mBringing ExecuTorch To the Next Frontiers of Edge AI
10mResearch to Production: PyTorch JIT/TorchScript Updates
33mExecutorch: Enabling and Accelerating PyTorch Models on Edge Devices
22mIntroducing ExecuTorch from PyTorch Edge: On-Device AI...
10mExecuTorch on Microcontrollers: Deploying PyTorch To...
13mLightning Talk: Accelerated Inference in PyTorch 2.X with Torch...
06mTorchVision
22mIncreasing the Impact of AI Through Better Software