The Future of Scala Performance on Big Data
summary
A Scala Big Data benchmark suite is open-sourced on GitHub that extracts hot methods from real workloads, runs time-based profiling on a laptop (no cluster), and compares JVM and Scala versions (Java7→8 shows ~5% hashmap gain). The driver automates warmup, multi-size runs, GC and deviance stats, and captures contention via Java Flight Recorder. Using the suite reveals a pathological pattern in the standard mutable hashmap and replaces long linked lists with an opportunistic red‑black tree so access complexity shifts from linear to logarithmic.
AI-written summary · about
chapters (13)
- 0:00Intro
- 0:50Intel and Scala
- 2:21Performance
- 4:11New benchmarks
- 5:34Apache Spark
- 6:15Real World Workload
- 7:25Benchmarks
- 9:17Benchmark Driver
- 11:37Example Output
- 13:01Example Benchmark
- 14:55Running the Benchmark
- 16:01What we have achieved
- 17:19Hashmap optimization
related talks
46mScala and the JVM as a Big Data Platform
48mHigh performance Scala systems without tears
49mNumbers every Scala programmer should know
01h 12mBig Data in Java World: Apache Spark and Scala. Arch and live coding
47mScala: The Unpredicted Lingua Franca for Data Science
54mHow Not to Write Slow Scala
34mBig Data, Small Code ... #bbuzz
29mBig Data Analysis