Towards Parallelizing Scala Compilations

on YouTube ↗26m

" summary

Rsc computes public type signatures called outlines, dumps them in the Scala compiler format, and then launches multiple scalac instances to compile partitions in parallel using a toy build tool called warp. On Twitter Util (75k LOC, 60 targets) running on dual 16-core machines this multi-phase approach reduces end-to-end build time from ~37s to ~6s — about 2.7x overall speedup. A limited Rsc prototype type checker shows 20–25x speedups on simple inputs and outlining can be ~10x faster than scalac compilation in practice.

AI-written summary · about

" chapters (6)

1. 0:00Introduction to parallel Scala compilation
2. 2:51What is RSC and its goals
3. 9:54From prototype to practical use
4. 13:55Multi-phase compilation explained
5. 20:26Experimental results and benchmarks
6. 24:47Credits and future work

" related talks

• 31mRsc: Scala Outlining for Distributed CompilationWin Wang · Scale By The Bay 2019
• 48mMassively Parallel Distributed Scala Compilation And YouStu Hood · Scala Days Lausanne 2019
• 21mReasonable Scala CompilerEugene Burmako · Scale By the Bay 2017
• 31m5 Things you Need to Know About Scala CompilationIulian Dragos · Scala Days New York 2018
• 46mCompiling Scala Faster with GraalVMVojin Jovanovic · Scala Days Berlin 2018
• 32mSpeedy Scala Builds at DatabricksLi Haoyi, Ahir Reddy · Scale By The Bay 2019
• 57mScalan: a reasonably typed meta programming framework in ScalaAlexander Slesarenko · Scala Days Berlin 2016
• 01h 05mScala: The Simple PartsMartin Odersky · GOTO Chicago 2014