Next-generation frameworks for Large-scale Machine Learning
summary
Argues that keeping multidimensional structure yields practical gains: tensor layers (tensor-train and convolutional tensor-train) compress fully connected and recurrent blocks, cut parameters by up to ~65% while improving long-range forecasting (sharp video prediction up to ~38 frames) and smaller models beat baselines. Shows simple gradient compression (SignSGD one-bit quantization) across multi-node training preserves accuracy while reducing communication. Mentions open-source tooling like testily (PyTorch/TensorFlow backends) and kuTensor CUDA kernels to speed tensor primitives.
AI-written summary · about
chapters (14)
- 0:00Introduction to next-gen ML
- 1:28AI as a data-infrastructure trinity
- 3:15Data hunger and inductive bias
- 5:23Tensor layers as new primitives
- 9:23Tensors in deep learning
- 14:04Tensor layers in CNNs
- 16:56Tensor layers for sequences
- 19:05Tensor layers for video prediction
- 21:18TensorFlow and PyTorch integration
- 22:49Tensors and GPU acceleration
- 23:32Gradient compression benefits
- 25:42End-to-end pipeline optimization
- 26:46Simulation for real-world testing
- 27:08Future of ML frameworks
related talks
24mKeynote: Next-generation frameworks for Large-scale AI
45mTensorly: A Flexible Python Framework for Machine Learning
01h 14mDistributed and concurrent optimization for machine learning
30mCompactifAI: Quantum-Inspired AI Model Compression
19mOptimizations and abstractions for sparse machine learning
32mTaming the Machine: Basics of ML Models Training and Inference Optimization
30mUnderstanding GPUs
36mHigh-Performance Distributed Tensorflow