Clojure Remixed: Pyroclast Data Processing
summary
Pyroclast is a hosted streaming platform built on Onyx that compiles pipeline definitions (cljc) into Onyx jobs and runs a local ClojureScript simulator in the browser that re-executes a shadow distributed program on every keystroke (target <16ms, falls back to a web worker). Demo shows file-mode with 198 sample records, a global sum of 9,924 points, fixed windows and session windows, Datomic for metadata, DynamoDB for window storage, Kubernetes for runtime, and exposed microservice CQRS endpoints with constant-time reads.
AI-written summary · about
chapters (33)
- 0:00Intro
- 4:20Getting it right
- 5:56Pyroclast
- 6:39Streaming
- 7:21Why
- 7:42Four Pillars
- 8:37Ripples
- 12:17Horizontal stripes
- 12:37Pyroclast simulator
- 14:22Demo
- 15:34Service
- 17:51Deep introspection
- 18:12Highlighting blue
- 19:00Moving messages
- 19:54Conch
- 20:39Sample Data
- 21:06Sample Data File
- 21:17HTTP Endpoint
- 22:49Analytics
- 23:36Time Ranges
- 24:32Group by Key
- 25:04Session
- 26:59Session Growth
- 28:18Backing Data
- 29:22Closure Script
- 30:35Compiler
- 31:24Data Structure
- 32:00Execution Environment
- 32:37Never talks to the Internet
- 33:37Onix
- 35:00Architecture
- 35:19Multiple representations
- 35:56Compilers
