Phantora: Maximizing Code Reuse in Simulation-based Machine Learning...
summary
Phantora runs unmodified ML frameworks inside containerized rank emulators and intercepts CUDA/NCCL calls to forward computation and communication events to a discrete-event simulator, using one real GPU for kernel profiling. It supports Megatron, DeepSpeed and TorchRec out of the box, simulates up to 256 GPUs at roughly one minute per GPU per iteration, and reports 2.9% average error (8.5% max) versus published benchmarks using hybrid simulation and one-GPU profiling.
AI-written summary · about
chapters (10)
- 0:00Phantora introduction and goals
- 1:44Simulation method limitations
- 4:53Hybrid simulation code reuse
- 6:57Phantora architecture components
- 8:21Rollback for past events handling
- 10:50Emulating CUDA PyTorch behaviors
- 12:13Transparent execution tuning case
- 13:39Validation Touch Titan benchmarks
- 14:43Future directions challenges
- 16:27Conclusion open-source
related talks
36mAccelerating High-Performance Machine Learning at Scale i...
26mThunder: Distribute and Optimize Your PyTorch Models With Zero...
16mAccelerating Design Space Exploration for LLM Training Systems with Multi-experiment...
11mSimAI: Unifying Architecture Design and Performance Tuning for Large-Scale Large...
17mServing Heterogeneous Machine Learning Models on Multi-GPU Servers...
26mDeepSpeed: Efficient Training Scalability for Deep Learning
32mTaming the Machine: Basics of ML Models Training and Inference Optimization
01h 18mOptimizing LLM Training and Inference Performance on GPUs