PyTorch Conference 2023
2023-10-16..2023-10-17
San Francisco, United States
[PyTorch Conference 2022, PyTorch Conference 2023, PyTorch Conference 2024, PyTorch Conference 2025]
as the organisers put it
sponsor register view the schedule view the schedule Submit a Proposal Thank you to all who joined us for PyTorch Conference 2023! We look forward to seeing you at future events.
who it's for
PyTorch developers, machine learning engineers, research engineers, ml platform engineers, infrastructure engineers, MLOps engineers, edge developers
topics
torch.compile and torch.func; MPS backend for Apple GPUs; Triton GPU kernel compiler; CUTLASS for NVIDIA Tensor Cores; ONNX exporter via torch.onnx.dynamo_export; Torch-TensorRT integration with Dynamo; PyTorch on ROCm and AMD support; XLA SPMD and DTensor partitioning; quantized offload to Hexagon NPU; continuous benchmarking with TorchBench
| title | author | length |
|---|---|---|
| Lightning Talk: State of PyTorch | Alban Desmaison | 12m |
PyTorch announces concrete updates: PyTorch 2.0 reaches >20 million downloads and ships torch.compile plus torch.func, the MPS backend enters beta for Apple… | ||
| Lightning Talk: TorchFix - a Linter for PyTorch-Using Code with Autofix Support | Sergii Dymchenko | 09m |
A PyTorch-focused linter with autofix support analyzes code with LibCST to find and rewrite concrete issues: deprecated API moves… | ||
| What's New for PyTorch Developer Infrastructure | Eli Uriegas, Omkar Salpekar | 21m |
Updates present a concrete developer-infra stack: hud.pytorch.org timeline for CI signal, a GitHub-first bot merge implementation, automated flaky-test… | ||
| Lightning Talk: Enhancements Made to MPS Backend in PyTorch for Applications Running... | Kulin Seth | 08m |
MPS backend qualifies as beta in PyTorch 2.0 with support for 300+ operators and expanded network coverage; profiling is integrated via OS signpost and Metal… | ||
| PyTorch Korea User Group: The Beginning, Present, and Future | Junghwan Park | 25m |
Community launched in 2018 and grows into a self-sustaining ecosystem that runs translated tutorials, a GitHub org and Discord, and hosts recorded sessions on… | ||
| Lightning Talk: Triton Compiler | Thomas Raoux | 16m |
Triton is a Python DSL and compiler that generates GPU kernels shown to reach near‑CUDA performance with far less code, and it ships as a fundamental part of… | ||
| Lightning Talk: Harnessing NVIDIA Tensor Cores: An Exploration of CUTLASS & OpenAI.. | Matthew Nicely | 09m |
CUTLASS v3 targets NVIDIA Tensor Cores with a layered C++ library and a new Python front end that cuts boilerplate to about one third the parameters, adds an… | ||
| Lightning Talk: PyTorch 2.0 on the ROCm Platform | Douglas Lehr | 12m |
PyTorch on ROCm gains day 0 Triton support for 2.0 with no API changes required, plus hipGraph capture, hipSolver, and DOT graph dumps; Triton adds MF (matrix… | ||
| Lightning Talk: Accelerated Inference in PyTorch 2.X with Torch... | George Stefanakis, Dheeraj Peri | 13m |
Torch‑TensorRT uses PyTorch 2.x Dynamo to convert models into TensorRT engines either just‑in‑time via torch.compile(model, backend="tensorrt") (handles… | ||
| Lightning Talk: Large-Scale Distributed Training with Dynamo and... | Yeounoh Chung, Jiewen Tan | 14m |
Experimental PyTorch/XLA SPMD uses XLA GSPMD to transform single-device PyTorch programs into partitioned ones with mark_sharding annotations and integrates… | ||
| Lightning Talk: Streamlining Model Export with the New ONNX Exporter | Maanav Dalal, Aaron Bockover | 10m |
New exporter (beta in PyTorch 2.1) uses TorchDynamo to produce ONNX from dynamic PyTorch graphs via torch.onnx.dynamo_export and runs with ONNX Runtime. It… | ||
| Lightning Talk: Efficient Inference at the Edge: Performance You Need at the Lowest... | Felix Baum | 16m |
Qualifies a PyTorch 2.0 workflow integrated with the Qualcomm AI Stack and an executortorch delegator that AOT compiles quantized subgraphs and offloads them… | ||
| Lightning Talk: Accelerating LLM Training on Cerebras Wafer-Scale... | Natalia Vassilieva, Behzad Abghari, Emad Barsoum, Mark Browning | 10m |
Cerebras builds a wafer-scale cluster with an 850,000‑core WSE, 80 GB L1 cache, 1.2 Tb optical IO and 28 kW chassis that treats a 1–128 CS2 cluster as a single… | ||
| Lightning Talk: Accelerating PyTorch Performance with OpenVINO | Yamini Nimmagadda, Mustafa Cavus, Devang Aggarwal | 11m |
OpenVINO loads PyTorch models either by convert_model -> OpenV IR (inline graph optimizations) or as a PyTorch 2.0 torch.compile backend (TorchDynamo backend… | ||
| PyTorch Edge: Developer Journey for Deploying AI Models Onto Edge Devices | Mengwei Liu, Angela Yi | 20m |
Walks through export → lowering → AOT compile into a PTTE binary and an edge runtime that runs on phones, embedded devices and VR headsets. Claims quantization… | ||
| PyTorch Edge: Vendor Integration Journey for Compilers and Backends | Kimish Patel, Chen Lai | 25m |
Describes concrete integration points for PyTorch Edge: a quantization API with prepare/calibrate/convert and an annotate() quantizer that emits dquan/quan… | ||
| Lightning Talk: The Fastest Path to Production: PyTorch Inference in Python | Mark Saroufim | 14m |
Describes a concrete path to run PyTorch in production using torch.compile with new APIs to cache compilation artifacts across devices and python bindings for… | ||
| Lightning Talk: Exploring PiPPY, Tensor Parallel and Torchserve for Large... | Hamid Shojanazeri | 13m |
Covers how Torchserve runs large-model inference using PiPPY pipeline parallelism and tensor parallel APIs plus integrations with DeepSpeed, Hugging Face… | ||
| Lightning Talk: Standardizing CPU Benchmarking with TorchBench for PyTorch... | Xu Zhao, Mingfei Ma | 08m |
TorchBench now includes a standardized CPU harness that runs FX INT8 and torch AMP (bf16/fp32) on CPU, compares channel_last (NHWC) vs channel_first formats… | ||
| Lightning Talk: Profiling and Memory Debugging Tools for Distributed ML Workloads on GPUs | Aaron Shi | 10m |
Overview of concrete debugging steps and outcomes for PyTorch GPU memory: introduces Memory Snapshot that records allocation stacks and produces time-series… | ||
| Lightning Talk: Building Intermediate Logging for PyTorch | Kunal Bhalla | 12m |
An API captures module arguments, parameters, buffers, outputs and gradients that works across TorchScript, torch.compile, Torch FX, torch.package and plain… | ||
| Lightning Talk: PT2 Export - A Sound Full Graph Capture Mechanism for PyTorch | Avik Chaudhuri | 15m |
Introduces torch.export() as an ahead-of-time full-graph capture for PyTorch that produces an FX graph plus source metadata and static/dynamic shape info, with… | ||
| Llama V2 in Azure AI for Finetuning, Evaluation and Deployment from the Model Catalog | Swati Gharse | 23m |
Llama 2 models (pretrained on about two trillion tokens) appear in the Azure ML model catalog as ready-to-run artifacts with 7B, 13B, 70B sizes and fine-tuned… | ||
| Cost Effectively Deploy Thousands of Fine Tuned Gen AI Models Like... | Saurabh Trikande, Li Ning | 21m |
Shows how to run thousands of fine‑tuned Llama models on shared GPU/CPU instances using TorchServe and SageMaker Multi-Model Endpoint with PEFT for… | ||
| The Evolving Landscape of Dataloading | Laurence Rouesnel | 24m |
PyTorch data loading is fragmented and increasingly the tight spot for large training jobs: attempts to ship a one‑size library paused torchdata effort in July… | ||
| Lightning Talk: Adding Backends for TorchInductor: Case Study with Intel GPU | Eikan Wang | 13m |
Two integration levels exist: ATen/PrimsIR and the lower Inductor loop IR, and a dynamic registration mechanism is implemented so a backend can register… | ||
| Keynote: Welcome & Opening Remarks – Ibrahim Haddad | Ibrahim Haddad | 06m |
Conference opens by noting attendance growth from about 200 to roughly 600 people and thanking event sponsors; foundation reports founding six members (Amazon… | ||
| Keynote: How PyTorch Became the Foundation of the AI Revolution | Joe Spisak | 12m |
PyTorch claims 20% year‑over‑year growth, enables 600,000 projects, and records nearly 276 million downloads since 2021, with roughly 3,000 contributors since… | ||
| Keynote: PyTorch 2.1 Technical Deep Dive | Mark Saroufim, Mergen Nachin, Joe Spisak, Peng Wu, Will Constable, Yanan Cao, Mario Lezcano | 54m |
PyTorch 2.1 shows concrete gains: torch.compile reaches broad adoption and reduces graph breaks to boost single‑GPU training 1.8x training on 180+ models and… | ||
| What's New for Dynamic Shapes in PyTorch 2.1 | Edward Yang | 24m |
PyTorch 2.1 moves dynamic shapes to Beta: dynamic detection is now automatic (first compile specializes, later runs upgrade changing dims), and training with… | ||
| Lightning Talk: CUDAGraph in a Partial Graph World | Elias Ellison | 12m |
CUDAGraph trees integrate with torch.compile to replace per-graph private pools with a single CUDA memory pool that recreates eager allocation sequences… | ||
| Lightning Talk: AOTInductor: Ahead-of-Time Compilation for PT2 Exported Models | Bin Bao | 15m |
AOTInductor ahead-of-time compiles torch.export graphs into a ahead-of-time compiled shared library containing C++ runtime glue and precompiled cuda cubin… | ||
| Lightning Talk: Accelerating Inference on CPU with Torch.Compile | Jiong Gong | 12m |
Torch.compile CPU backend generates C++ fusion kernels, supports fp32, bfloat16 and int8, and runs dynamic-shape models like GPTJ and Llama with AMX on… | ||
| Lightning Talk: Lessons from Using Pytorch 2.0 Compile in IBM's Watsonx.AI Inference | Antoni Martin | 11m |
Open source LLaMA implementation for Watsonx.AI uses torch.compile() plus SDPA, tensor parallel and CUDA graphs and shows at least a 2X improvement on many… | ||
| Keynote: Welcome & Opening Remarks – Joe Spisak | Joe Spisak | 14m |
Over 100 projects in the PyTorch ecosystem and 317,000 GitHub repos depend on it, and the talk lists five concrete priorities: well tested with green CI, clear… | ||
| Keynote: Refik Anadol Studio: Rainforest AI Research | Christian Burke, Refik Anadol | 16m |
Studio presents custom PyTorch models that generate Amazon fauna, flora and funga and a Large Nature Model trained on field captures from 16 sites plus drones… | ||
| Keynote: AMD & PyTorch: A Powerful Combination for Generative AI | Negin Oliver | 03m |
AMD presents concrete support for PyTorch through the ROCm open stack and claims PyTorch 2.0 day-zero support with upstreamed features like torch.export… | ||
| Keynote: Building an Interoperable Ecosystem for Generative AI | Stella Biderman | 11m |
Open-source generative AI adoption hinges on concrete interoperability: models must be published to Hugging Face Hub to get noticed, training stacks like GTB… | ||
| Keynote: The Promise of PyTorch as a General-Purpose Array-Oriented Computational.. | Travis E. Oliphant | 05m |
Argues that PyTorch now serves as a general-purpose array backend by implementing the Array API, so libraries like scikit-learn and SciPy can dispatch to a… | ||
| Keynote: How to Leverage PyTorch to Scale AI Training and Inferencing | Raghu Ganti | 10m |
IBM shows concrete scaling steps using FSDP, adding a rate‑limiter API, S3 distributed checkpointing, and hybrid sharding to cut network traffic. Training on… | ||
| Keynote: The Value of Open Source for the Enterprise | Priya Nagpurkar | 05m |
IBM commits PyTorch as a core of Watson X to standardize inference, tuning and training with reusable building blocks that let platforms optimize across the… | ||
| Keynote: Intel and PyTorch: Enabling AI Everywhere with Ubiquitous Hardware and Open... | Fan Zhao | 06m |
Intel positions a full-stack AI offering: Xeon CPUs with AI features, Habana Gaudi2 for dedicated deep learning, and data-center GPUs plus client CPU/GPU/MPU… | ||
| Keynote: PyTorch Lightning: Powering the GenAI Revolution from Research to the... | William Falcon | 06m |
PyTorch Lightning 2.1 arrives with concrete large‑model features: expanded Trainer and Fabric sharding and multi‑device orchestration, plus tools for… | ||
| Keynote: The Llama Ecosystem: Past, Present and Future | Joe Spisak | 12m |
Llama begins as a scrappy Paris research project and scales to 7B, 13B, 70B models with an instruction-tuned Llama 2 and Code Llama checkpoints (7, 13 and 34B)… | ||
| Accelerating Explorations in Vision and Multimodal AI Using Pytorch... | Philip Bontrager, Peng Chen, Evan Smothers, Nicolas Hug | 26m |
TorchMultimodal adds native diffusion primitives so developers can build a Dolly 2 generator in five lines using an ADM U-Net plus stackable adapters like… | ||
| Getting Started with Pytorch 2.0 and Hugging Face Transformers | Philipp Schmid | 20m |
Fine-tunes BERT-base-uncased on Banking77 (13k examples, 77 classes) using Hugging Face Transformers and shows that flipping one flag plus a faster optimizer… | ||
| Training a LLaMA in your Backyard: Fine-tuning Very Large... | Sourab Mangrulkar, Younes Belkada | 22m |
Shows how to fine-tune LLaMA 7B on consumer GPUs using LoRA, QLoRA and Hugging Face PEFT/Transformers to cut memory: full fine-tune needs ~112 GB, LoRA reduces… | ||
| Distributed Checkpoint | Iris Zhang, Chien-Chin Huang | 15m |
Distributed checkpointing saves and loads model and optimizer shards in parallel, writes one file per rank plus a metadata file, and performs load-time… | ||
| Composable Distributed PT2(D) | Wanchao Liang | 19m |
PyTorch Distributed adds three composable pieces: Device Mesh for automatic process-group setup across hosts and GPUs, DTensor for tensor-level… | ||
| Lessons Learned in WatsonX Training: Scaling Cloud-Native... | Davis Wertheimer, Supriyo Chakraborty | 21m |
Claims PyTorch FSDP and a hybrid HSDP/DDD approach scale model pretraining from 100M to 20B parameters using up to 512 GPUs, training on >10 TB of web data… | ||
| Women and Non-Binary in PyTorch Breakfast with Quiana Berry, Red Hat | Quiana Berry | 41m |
Covers concrete harms and fixes: cites 95% non-consensual deepfakes and $78B misinfo cost (2020), warns AI energy use could match the Netherlands by 2027 and… | ||
| Lightning Talk: Tensor Query Processing | Matteo Interlandi | 16m |
Tensor Query Processor (TQP) maps SQL physical plans into PyTorch tensor programs, implements custom tensor operators for joins and group-by, and applies… | ||
| Lightning Talk: Energy-Efficient Deep Learning with PyTorch and Zeus | Jae-Won Chung | 13m |
Zeus integrates with PyTorch to measure GPU time and energy via a Zeus monitor, and automatically tunes GPU knobs (power limit and SM frequency) with a power… | ||
| Lightning Talk: Tensor and 2D Parallelism | Rodrigo Kumpera, Junjie Wang | 11m |
Enables DDP + tensor parallel and a prototype pipeline + tensor parallel with activation-checkpointing that all-gathers shared inputs during recompute; adds… | ||
| Lightning Talk: Seismic Data to Subsurface Models with OpenFWI | Benjamin Consolvo | 10m |
Neural network directly maps seismic shot gathers to velocity models using the OpenFWI InversionNet UNet-style architecture trained on roughly 150,000 shot… | ||
| LightningTalk: MultiRay: An Accelerated Embedding Service for Content... | Michael Gschwind | 14m |
MultiRay is an embedding service that computes and caches encoder-based embeddings (XLM-R and successors) for text, image and multimodal posts, exposing… | ||
| Lightning Talk: Uplink Interference Optimizer, How to Optimize a Cellular Network... | Oscar Gonzalez | 12m |
A pretrained graph neural network built with PyTorch and PyTorch Geometric models uplink SINR and lets parameter search run offline so live networks never need… | ||
| Accelerating Generative AI | Christian Puhrsch, Horace He | 27m |
Native PyTorch pipelines compile and fuse real kernels to cut overhead and hit near memory bandwidth limits: baseline LLaMA‑7B goes from ~25 to 107 tokens/s… | ||
| Lightning Talk: TorchRL - RLHF Support | Vincent Moens | 18m |
Adds RLHF primitives to TorchRL that let teams assemble training pipelines (model pretraining, reward-model training, RL fine-tuning) instead of using a single… | ||
| Lightning Talk: Diffusers: Bringing Cutting-Edge Diffusion Models to the Masses | Lysandre Debut | 10m |
Diffusers is a PyTorch library that loads Hugging Face checkpoints to run image, video, audio and 3D diffusion models with minimal code — examples generate an… | ||
| TorchBench: Guarding the Performance of the PyTorch Ecosystem with Continuous Benchmarking | Xu Zhao | 25m |
Continuous benchmarking for PyTorch that runs a reproducible model suite (grown from ~50 to over 113 models) with a Docker image tied to each nightly release… | ||
| Lightning Talk: Simulating Quantum Systems with PyTorch | Pierre Guilmin, Alice und Bob | 12m |
PyTorch-powered dynamiqs (github.com/dynamiqs/dynamiqs) provides GPU-accelerated, differentiable solvers that implement the Schrödinger Equation, Lindblad… | ||
| Lightning Talk: A Novel Domain Generalization Technique for Medical Imaging Using... | Dinkar Juyal | 09m |
A PyTorch module called ContriMix disentangles biological content and technical attributes, mixes them to generate synthetic patches, and plugs into training… | ||
| Lightning Talk: Leveraging PyTorch 2.0 for Bias Reduction in AI | Christina Zhu | 12m |
Lightning summary: shows concrete steps using PyTorch 2.0 features and third‑party tools to detect and reduce bias — cites the 2018 Gender Shades result (error… | ||
| Lightning Talk: Dinosaur Bone Hunt | Bob Chesebrough | 10m |
Builds a probabilistic heat map from tiled aerial images (224x224–512x512) using ResNet18 trained in PyTorch and accelerated with the Intel AI Analytics… | ||
| Lightning Talk: Orchestrating Machine Learning on Edge Devices... | Rishit Dagli, Shivay Lamba | 12m |
Claim: PyTorch models can run efficiently on heterogeneous edge devices by compiling inference to WebAssembly, scheduling them via a Wasm broker, and treating… | ||
| Into Generative AI with PyTorch Lightning 2.0 | Luca Antiga, Carlos Mocholí, Adrian Walchli | 25m |
PyTorch Lightning 2.0/2.1 is presented as the practical stack for generative AI: Lightning Fabric unbundles the trainer for custom loops, Lit‑Llama/Lit‑GPT… | ||
| Introducing ExecuTorch from PyTorch Edge: On-Device AI... | Mergen Nachin, Orion Reblitz-Richardson | 22m |
ExecuTorch launches as a PyTorch Edge on‑device stack that exports models with torch.export, compiles ahead‑of‑time, and delegates subgraphs to Arm Apple and… | ||
| nothing matches | ||