Evolving an Actor Library Based on Lessons Learned from Large-Scale Deployments

on YouTube ↗01h 01m

" summary

Built a pre‑C++11 actor+future library called libprocess to scale Apache Mesos to 75,000 physical machines, finds the implementation causes excessive synchronization and dynamic allocations, and rebuilds it as eventuals that use continuations (lazy, coroutine‑friendly) instead of eager futures/promises. The new design shows how continuations compose, how scheduling binds continuations back to the prior executor, why actor mailboxes avoid user locks, and where memory and execution resumption must be allocated or pinned for performance.

AI-written summary · about

" chapters (45)

1. 0:00Introduction
2. 1:08Chapters
3. 1:54Motivation
4. 2:12Challenges
5. 2:34Motivation Example
6. 2:48Function Composition
7. 3:31Wait
8. 4:00Threads
9. 4:30Code routines
10. 5:15Callbacks
11. 5:41Futures Promises
12. 7:16Future Promises
13. 9:03Simple Function Composition
14. 11:00Futures and Promises
15. 11:41Grammar Check
16. 13:20Waiting
17. 13:31Concurrency
18. 14:18Thread Pool
19. 14:53Multiple Problems
20. 15:12Solutions
21. 16:22Actors
22. 17:50Poltergeist
23. 18:17C Actor Libraries
24. 24:19C Star Project
25. 25:38Why Actors Need Futures
26. 26:10Control Flow
27. 27:09Transforming Actors
28. 30:55mutexes
29. 31:57asynchronous mutexes
30. 34:06Evolving the library
31. 36:42Revisiting the problem
32. 38:00Why do we need locks
33. 40:14And can we avoid dynamic allocation
34. 40:59Using callbacks everywhere
35. 43:30Cancellation paths
36. 43:43Memory
37. 43:58Memory Allocation
38. 45:39Continuation
39. 48:41Eventuallys
40. 50:05Eventually
41. 50:34Post
42. 51:07SpellCheck
43. 51:29GrammarCheck
44. 52:22Future
45. 59:30Conclusion

" related talks

• 43mIntroduction to Akka Actors: Concurrency and Distributed Systems for the JVMGuillaume Bogard · ScalaIO 2018
• 34mType-safe & high-perf distributed actor systems with RustAnselm Eickhoff · RustFest Zürich 2017
• 40mActors in .NET, 2025 editionVagif Abilov · Øredev 2025
• 24mUsing an actor framework for scientific computingKrzystof Borowski · Lambda Days 2016
• 58mLife with actors: experience reportVagif Abilov, Erlend Wiig · NDC London 2017
• 43mEffective Eventual Consistency with Actor Models + Amazon Web ServicesPhilip Laureano · NDC Sydney 2017
• 45mReactive design patterns for microservices on multicoreCharly Bechara, Alain Habra · Voxxed Days Microservices 2018
• 42mLeveraging the Actor Model in Financial Transaction SystemsNikita Melnikov · Conf42 Platform Engineering 2024