TorchCodec: The Media Decoding Library for PyTorch
summary
A PyTorch library that decodes video and audio into tensors, available via pip install torchcodec, wraps FFmpeg to provide CPU and CUDA decoding and returns pure uint8 tensors plus metadata (pts, duration). It offers index and time-based APIs, per-frame and batched frame objects, clip samplers, and file-like or URL streaming that downloads only needed chunks. A lightweight scan builds keyframe indexes but can be skipped with an approximate mode for much faster startup; audio decoding is already supported and encoders are planned.
AI-written summary · about
chapters (9)
- 0:00TorchCodec intro
- 1:46Core features and goals
- 3:12Metadata handling
- 5:59Index and time APIs
- 7:23Clip sampling
- 8:46Performance tips
- 10:12Audio decoding
- 10:55Migration from TorchVision
- 11:37Roadmap
related talks
27mVideo Editing in the Browser
19mOn Dynamic Multimedia Pipelines for Opening Browsers to New Formats
15mMediaPipe Web: Bringing cross-platform AI tech to the browser
08mReconstructing 3D from Compressed Video: An AV1-Based SfM Pipeline
46mDSP Welcomes Open Source - Xtensa HIFI DSP Support in Clang1
47mVideo transcoding on the web
43mTensorflow on open source GPUs
33mTranslate tech talks using open-source models and cloud-native Wasm