Spores: Distributable Functions in Scala
summary
A new Scala pickling framework and a closure discipline called Spores aim to make function shipping practical: pickling uses macro‑based generation and type classes to inline serializers and achieves about 6× speedup over Java Serialization and up to 3× faster than Kryo in microbenchmarks while reducing memory and serialized size; Spores are composable mobile closures with explicit captured environments and optional type exclusion constraints, enabling safer Spark/Akka style pipelines and a prototype proposed for Scala 2.11.
AI-written summary · about
chapters (17)
- 0:00Introduction
- 1:21Making distributed programming easier
- 1:57About the projects
- 4:11Function literals
- 5:01Moving closures
- 7:13Functional programming
- 8:50mainline spores
- 17:06four comprehensions
- 20:44new patterns
- 23:15spore type constraints
- 29:44oopsle
- 30:36pickling
- 33:18benefits
- 34:26pickle
- 34:57benchmarks
- 37:18more benchmarks
- 40:05questions
related talks
54mHow to avoid safety hazards when using closures in Scala
31mFunction-Passing Style Typed, Distributed Functional Programming
17mHerding types with Scala macros
01h 05mScala: The Simple Parts
42mSpeck flavored streaming micro services with Akka
45mScala and the JVM as a Big Data Platform Lessons from Apache Spark
45mEffective Scala
47mScala: The Unpredicted Lingua Franca for Data Science