Live Q&A Session 2
summary
Panel answers focus on concrete engineering tradeoffs: scale means training huge models, cost-effective serving, and infusing models across apps; current PyTorch compiler iterations improve hotspots but no single solution is yet battle tested; large-model inference needs composed compression and engineering (example: Copilot optimizations); production pain points are lack of visibility and training/serving skew so teams must deploy model monitoring via distribution-shift detection; recommended investment is an open spec for inference DAGs and robust orchestration with backend plugins.
AI-written summary · about
chapters (12)
- 0:00Host introduction
- 0:26Presenters round table
- 2:56AI scale definition
- 6:31AI scaling bottlenecks
- 9:42PyTorch compiler impact
- 11:48Efficient inference optimization
- 15:20Distributed training monitoring
- 19:12Training-serving skew
- 24:24AI DAG engines
- 27:14Model degradation detection
- 29:44Synthetic data impact
- 32:12Event wrap-up
