EINNET: Optimizing Tensor Programs with Derivation-Based Transformations

on YouTube ↗15m

" summary

EinNet claims a larger optimizer search space by turning tensor programs into tensor expressions and applying derivation-based tensor expression transformations to create new operators and rewrites, then dispatching work via an operating machine that maps parts to vendor libraries or kernel generators. Implemented as a compiler optimizer and runtime, it speeds up seven DNNs by up to 2.72× on A100 (1.52× average) and 2.68× on V100 (1.55× average), with code on GitHub.

AI-written summary · about

" chapters (9)

1. 0:00EINNET introduction
2. 2:07Prior optimizer limits
3. 3:303x3 convolution split example
4. 7:06Expression derivation rules
5. 8:55Execution strategies
6. 10:43EINNET workflow
7. 12:08Search overhead handling
8. 13:34Evaluation setup & results
9. 14:15Conclusion

" related talks

• 15mPET: Optimizing Tensor Programs with Partially Equivalent Transformations and AutomatedZhihao Jia, Shizhi Tang, Liyan Zheng, Jidong Zhai, Mingyu Gao, Haojie Wang, Zixuan Ma, Yuanzhi Li, Kaiyuan Rong, Yuanyong Chen · 15th USENIX Symposium on Operating Systems Design and Implementation (OSDI '21)
• 11mQiMeng-Xpiler: Transcompiling Tensor Programs for Deep Learning Systems with a...Qi Guo, Xuehai Zhou, Shouyang Dong, Yuanbo Wen, Jun Bi, Di Huang, Jiaming Guo, Jianxing Xu, Ruibai Xu, Xinkai Song, Yifan Hao, Ling Li, Tianshi Chen, Yunji Chen · OSDI '25
• 37mDeep Learning Neural Network Acceleration at the EdgeAndrea Gallo · Open Source Summit Europe & ELC + OpenIoT Summit Europe 2018
• 22mMonoNN: Enabling a New Monolithic Optimization Space for Neural Network Inference...Wei Lin, Haojun Xia, Zhen Zheng, Donglin Zhuang, Shuaiwen Leon Song, Xiafei Qiu, Junjie Bai · 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24)
• 37mDeep Learning Neural Network Acceleration at the EdgeAndrea Gallo · Open Source Summit Europe & ELC + OpenIoT Summit Europe 2018
• 15mEnabling Tensor Language Model to Assist in Generating High-Performance Tensor...Yu Zhang, Yi Zhai, Sijia Yang, Keyu Pan, Renwei Zhang, Shuo Liu, Chao Liu, Zichun Ye, Jianmin Ji, Jie Zhao, Yanyong Zhang · 18th USENIX Symposium on Operating Systems Design and Implementation (OSDI '24)
• 26mNeuOS: A Latency-Predictable Multi-Dimensional Optimization Framework for DNN-driveCong Liu, Soroush Bateni · USENIX ATC '20
• 25mConvergence of AI, Simulations and HPCAnima Anandkumar · Scale By the Bay 2021