Getting Started with Pytorch 2.0 and Hugging Face Transformers
summary
Fine-tunes BERT-base-uncased on Banking77 (13k examples, 77 classes) using Hugging Face Transformers and shows that flipping one flag plus a faster optimizer yields big gains: set torch.compile true and use the fused Adam optimizer to run on NVIDIA A10 on AWS and drop training from 696s to ~475s (≈41% faster), raise throughput from ~30 to ~65.5 samples/s, and produce a slightly higher eval F1 (~93%), with identical code otherwise.
AI-written summary · about
chapters (18)
- 0:00Introduction
- 1:25Session goals and agenda
- 2:07PyTorch 2.0 new features
- 3:54How TorchCompile works
- 4:59PyTorch 2.0 performance
- 5:41Hugging Face ecosystem
- 7:48Hugging Face Hub and datasets
- 8:51Fine-tuning setup
- 10:16Dataset loading and prep
- 11:21Text tokenization and processing
- 12:24Mapping labels
- 13:07Training evaluation metrics
- 13:50PyTorch 2.0 with Trainer
- 14:52Training arguments
- 16:18Training loop
- 17:02PyTorch 1.13 vs 2.0 performance
- 18:49PyTorch 2.0 in HF libraries
- 19:52Closing remarks
related talks
02h 10mIntro to Hugging Face: Fine-tuning BERT for NLP tasks
01h 09mReinventing Machine Learning with Transformers and Hugging Face
32mAn Introduction to Transfer Learning in NLP and HuggingFace
34mTransformers End to End Experience training...
01h 32mIntroduction to PyTorch
25mEasy Access to SOTA NLP Models with Ray and Hugging Face
40mPanel discussion "NLP"
01h 16mOptimizing workloads with Transformers and Diffusers