Massively Parallel Distributed Scala Compilation And You

on YouTube ↗48m

" summary

Quilia wires RSC outlining to a Bazel remote execution API so Pants snapshots source digests, Scoot or BuildFarm workers run many Zinc/RSC compiles, and Graal native images keep JVM startup low. A real library with 4.5M generated Scala lines shows end-to-end runs across 128 cores beating 8-core local builds and recent optimizations double observed throughput while simulations predict up to 16x speedups; remaining long tails trace to Java/Scala alternation.

AI-written summary · about

" chapters (16)

1. 0:00Introduction
2. 1:14Trajectory: Builds in the future?
3. 1:57Trajectory: Builds today
4. 6:41Trajectory: Improve the workflow transparently!
5. 7:47The Continuing Story
6. 12:43Quelea: Outlining
7. 16:26Quelea: Remote Execution: Client
8. 17:29Quelea: Remote Execution: Servers
9. 18:39Quelea: Remote Execution: Tools
10. 19:42Quelea: Remote Execution: Graal Native Image
11. 27:09Today's Result, Tomorrow's Reference
12. 30:27Likely optimizations
13. 35:02Next Steps for Quelea: Removing zinc?
14. 40:01The Rest of the Trajectory: Getting involved
15. 41:47Conclusion: The Team
16. 42:15Conclusion: We're Hiring!

" related talks

• 26mTowards Parallelizing Scala CompilationsEugene Burmako · Scale By The Bay 2018
• 31mRsc: Scala Outlining for Distributed CompilationWin Wang · Scale By The Bay 2019
• 32mSpeedy Scala Builds at DatabricksLi Haoyi, Ahir Reddy · Scale By The Bay 2019
• 47m5 Things You Need To Know About Scala CompilationMirco Dotta · Scala Italy 2018
• 21mReasonable Scala CompilerEugene Burmako · Scale By the Bay 2017
• 37mScala with BazelIttai Zeidman, Ahir Reddy, Oscar Boykin · BazelCon 2017
• 45mSimpler Scala Builds with Functional and Object-Oriented ProgrammingLi Haoyi · The Scala Workshop 2025
• 48mHigh performance Scala systems without tearsZahari Dichev · Scala Days Berlin 2018