torch.accelerator: A Unified, Device-Agnostic Runtime API for Stream-Based Accelerators
summary
Proposal introduces a unified, device-agnostic runtime in PyTorch 2.6 that maps one-to-one to existing device APIs so code needs minimal changes, with a C++ counterpart and a new query torch.accelerator.current_accelerator. Backends register via DeviceGuardImplInterface, streams and events become polymorphic through stream/event base classes, and device/stream guards reflect each other. Supports CUDA, XPU, MPS, HPU and private accelerators under the single-accelerator-per-host assumption; memory APIs remain incomplete and a unified allocator and test generalization are planned (PR 132204).
AI-written summary · about
chapters (7)
- 0:00Introduction to API
- 0:22Challenges and vision
- 5:48New accelerator API
- 7:47Runtime component abstraction
- 10:24Design philosophy and evolution
- 16:38Future work and improvements
- 17:27Example and conclusion
related talks
31mWrite Once, Run Anywhere: PyTorch’s Generalization Journey
23mHugging Face Accelerate: Making Device-Agnostic ML Training and Inference Easy...
05mTowards Device-agnostic PyTorch: Building Unified Infrastructure for a Multi-Backend Ecos...
04mBuild a Unified Heterogeneous AI Computing Ecosystem for PyTorch
13mLightning Talk: Challenges and Standardization in PyTorch Ecosystem...
12mLightning Talk: A Stable Limited LibTorch ABI? How?! (and Why?)
16mHardware acceleration for on-device Machine Learning
33mExecutorch: Enabling and Accelerating PyTorch Models on Edge Devices