Scala(ble) High-Performance Computing
summary
A Scala framework implements a biological-habitat simulator using Akka Actors with shard-based Akka Cluster deployment, custom serialization, grid-splitting workers, buffer-zone messaging and configurable conflict-resolution. Experiments show linear scalability to 53,568 cores, a fixed-grid run (~300 GB on Prometheus) drops from 8 h 22 m to 11 m, and weak-scaling increases runtime by 44% while total problem size grows ~14,000%. The system runs distributed, deterministic iterations and exposes runtime-configurable synchronization strategies.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 6:41Concurrent Implementation
- 8:46Cluster charting mechanism
- 9:29Research paper
- 10:39Results
- 12:01Implementation
- 15:41Experiments
- 18:23Future plans
- 19:13Question
related talks
36mData pipelines engineering made simple with Scala
40mScaling with Apache Spark (or a lesson in unintended consequences)
57mBig Data Pipelines With Scala - Part 2 of 2
25mScaling Up Genomics with Spark
32mContaining an Elephant
39mEvaluating Streaming Framework Performance for a Large-Scale...
43mBeyond Shuffling: Scaling Apache Spark
08mThe high performance stack