Using an actor framework for scientific computing
summary
Scaflow is an Akka-based library and engine that implements scientific workflows with event sourcing persistent actors, supports push and pull worker models, remote actor deployment and HTTP retry strategies, and runs a biological-pathway pipeline that downloads graph and text representations concurrently. Codebase is under 1,500 LOC and uses Scaflow Akka Cassandra for persistence; a microbenchmark shows about 450k messages 2–3s overhead when persisting events, and restore-from-events works for simulated failures while full node-power loss and production hardening remain experimental.
AI-written summary · about
chapters (9)
- 0:00Start
- 1:17Scientific workflow
- 3:52Actor Model difficulties
- 6:39Current workflow engines
- 9:21Concurrent computations in actor model
- 10:56Scalability in actor model
- 11:22Workflow persistent state
- 12:25Fault tolerance
- 15:51Real world example - scaling
related talks
58mLife with actors: experience report
51mComposing high performance process workflows with Akka Streams
41mAkka in Production: Our Story
55mExperience report on using Akka in production: pitfalls and rewards
56mAkka in Heterogenous Environments
48mThe three paths of Scaling with Akka
41mA tale of futures, actors & pipes: using Akka for large data migration
23mScaling functional multi-agent computations (...)