Chip floorplanning with deep reinforcement learning

on YouTube ↗10m

" summary

A deep reinforcement learning pipeline generates chip floorplans for Google TPUs that match or beat expert layouts and runs under six hours instead of weeks. An RL agent sequentially places macros, a force‑directed placer fills standard cells, and a masked action space enforces feasibility. Training uses 10,000 placements across five blocks and an edge-aware graph encoder that predicts wire length and congestion; production TPU layouts from Nature results tape out successfully.

AI-written summary · about

" chapters (11)

1. 0:00Intro
2. 0:24Key takeaways
3. 1:41Prior approaches to chip floorplanning
4. 2:16Chip floorplanning with reinforcement learning
5. 2:44We take a hybrid approach to placement optimization
6. 2:55Results on a TPU-v4 block
7. 5:32Reward model architecture and features Input Features Graph Cory Predictions
8. 6:08Label prediction results on test chips
9. 6:49Ariane (RISC-V) placement visualization Training policy from scratch
10. 7:49Convergence curve: training from scratch vs. fine-tuning
11. 8:33Comparisons with manual and SOTA baselines

" related talks

• 20mKeynote: ML for Systems and Chip DesignAzalia Mirhoseini · Ray Summit 2020
• 26mChatGPT and Minecraft's diamonds playing agents: Reinforcement Learning in a nutshell with sheeprlFederico Belotti · PyCon Italia 2023
• 26mKeynote: Artificial Innovation Accelerates Artificial IntelligenceDavid Patterson · Ray Summit 2020
• 32mByteDance's Platform for Reinforcement Learning from Human FeedbackLiguang Xie, Haibin Lin, Chi Zhang · Ray Summit 2024
• 17mScaling Reinforcement LearningJyotirmoy Sundi · Conf42 JavaScript 2025
• 17mRLinf: Flexible and Efficient Large-Scale Reinforcement Learning...Yuanqing Wang, Zhihao Liu, Junhao Hu, Yongji Wu, Chao Yu, Yuqing Xie, Kang Chen, Hao Lin, Zhen Guo, Si Xu, Hongzhi Zang, Quanlu Zhang, Chunyang Zhu, Zixiao Huang, Mingjie Wei, Ke Yang, Bo Dai, Zhexuan Xu, Jiakun Du, Xiangyuan Wang, Xu Fu, Letong Shi · 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26)
• 19mDynaRL: Flexible and Dynamic Scheduling of Large-Scale Reinforcement Learning TrainingYuanqing Wang, Yuchen Zhang, Yu Wang, Junhao Hu, Chao Yu, Hao Lin, Zhi Yang, Zhen Guo, Si Xu, Quanlu Zhang, Chunyang Zhu, Zixiao Huang, Bo Dai, Xu Fu, Boxun Li, Guohao Dai · 20th USENIX Symposium on Operating Systems Design and Implementation (OSDI '26)
• 37mTensorflow and deep reinforcement learning, without a PhDMartin Gorner · Devoxx Belgium 2017