Data Science in Scala with ScalaPy

on YouTube ↗32m

" summary

ScalaPy embeds a CPython interpreter and exposes Python libraries to Scala on JVM and Scala Native via JNA or native FFI, with heap synchronization, GIL handling, reverse interop and proxy-backed sequence conversion (release 0.4.0). Examples run TensorFlow on Colab GPUs and a CNN on MNIST reaches 98.97% accuracy. It provides dynamic APIs for quick experiments, optional generated type facades for static safety, and bi-directional function/sequence bridges that avoid full-copy overhead.

AI-written summary · about

" chapters (14)

1. 0:00Intro
2. 0:26data science has a split ecosystem
3. 2:17ScalaPy lets you use Python libraries from Scala code
4. 11:30embedding Python
5. 14:11cross-platform bindings
6. 16:17dynamically typed APIs
7. 17:34python interpolation
8. 19:47sending functions to Python
9. 23:20sending sequences
10. 25:03proxy collections
11. 26:20type mappings
12. 26:36type hierarchy
13. 26:55creating facades
14. 30:08in conclusion

" related talks

• 31mBringing the Jewels of the Python World to Scala with SparkHolden Karau · Lambda World Seattle 2018
• 47mScala: The Unpredicted Lingua Franca for Data ScienceDean Wampler, Andy Petrella · Scala Days Berlin 2016
• 47mScala: The Unpredicted Lingua Franca for Data ScienceDean Wampler · GOTO Chicago 2016
• 42mDevelopment of Data Science Ecosystem for RubyKenta Murata · RubyKaigi 2017
• 33mSolving the Scala Notebook ExperienceJeremy Smith, Jonathan Indig · Scale By The Bay 2019
• 32mCyfra: Scala on GPUSzymon Rodziewicz, Marcin Złakowski · Scalar 2026
• 45mScala and the JVM as a Big Data Platform Lessons from Apache SparkDean Wampler · Scala Days Berlin 2016
• 33mIodide and Pyodide: Bringing Data Science Computation to the Web BrowserMichael Droettboom · PyData NYC 2018