Onyx: Distributed Computing for Clojure
summary
Onyx is a Clojure-native distributed data platform that ships as ~6,000 lines of runtime code, implements a masterless protocol and a custom network overlay, and supports stateful streaming with punctuated streams. Programs deploy as a catalog of behaviors so you bind logic at runtime via a catalog of tasks and lifecycle hooks; state uses windows and triggers with BookKeeper and RocksDB for durable, idempotent updates. Benchmarks show millions of messages per second, it scales linearly until hardware limits, and multiple companies run it in production.
AI-written summary · about
chapters (19)
- 0:00Introduction
- 0:41Batching
- 1:32Streaming
- 3:50Elevator Pitch
- 4:55So What
- 5:56Storm
- 12:38Core API
- 13:05Structure
- 16:09Bolt
- 17:23Functions
- 19:17Sideeffects
- 21:10Life Cycles
- 22:35Coding Life Cycles
- 25:11Flow Conditions
- 27:06Windows
- 29:16Triggers
- 32:12Ecosystem
- 35:52Lucas Bradstreet
- 36:52Outro
related talks
37mInside Onyx
37mClojure Remixed: Pyroclast Data Processing
01h 15mEffective Programs - 10 Years of Clojure
31mClojure at 4,000 msg/s-What We Learned, Loved, and Loathed
30mRealtime Collaboration with Clojure
43mClojure in live sports television
24mClojask: Inviting Data Scientists to Distributed Computing
26mDistributed Computing in Clojure and Spark