Antimony: a real time distributed stream processing system
summary
Antimony is a Rust reimplementation of real-time stream processing that claims Twitter Heron-inspired architecture, Tokio.rs-based async IO, and a JSON topology of spouts and bolts deployed to per-topology masters and stream managers. Uploader asserts 2–3× throughput versus the team’s Apache Storm runs and reports >10× resource savings by avoiding JVM/Python-in-jar deployment and using containerized scheduling. Expect concrete design choices, execution model, deployment artifacts, and measured performance numbers rather than high-level marketing.
AI-written summary · about
chapters (22)
- 0:00Intro
- 1:03The plan of this talk
- 1:36The use case for a real time streaming
- 4:27Attackers / Defenders best friend
- 5:03Apache Storm
- 6:12A Streaming Job
- 6:58Some Terminology
- 7:42Example topology
- 8:38Physical Plan of topology
- 9:27Motivation for Antimony
- 10:49Some problems with that architecture
- 12:17Inside the Storm Worker
- 13:32Zookeeper Overload
- 14:12Some homegrown wisdom
- 14:37Two things happened
- 15:28Antimony Architecture
- 16:31Topology Master
- 16:54Stream Manager
- 18:05Define a topology
- 19:14antimony-cli
- 19:34The Name
- 20:43Thank You
related talks
40mStreamling: Lightweight, Extensible Streaming on DataFusion
42mSupercharge Your Workers with Storm
51mComposing high performance process workflows with Akka Streams
36mStream All The Things!
30mEvent Streaming and the Future of Applications
46mEvent-based Architecture & Implementations with Kafka & Atom
50mHigh Performance Network Programming in Rust
29mLessons Learned: Porting a Streaming Data Pipeline from Scala to Rust.