Metrics Stream Processing Using Riemann
summary
Claims ability to process millions of events per second using an in‑memory index keyed by host+service and protobuf over NIO. Configuration is Clojure-configured pipelines with reusable libraries, unit tests and a tap breakpoint for stream inspection. Events carry a TTL-based expiry so "service down" is detected when events expire; alerts use rollup and throttle to avoid floods. Scaling options include async bounded queues, Kafka for durability, and batching for downstreams like InfluxDB or OpenTSDB; Grafana datasource exists.
AI-written summary · about
chapters (11)
- 0:00Introduction to Riemann
- 1:53Riemann core concepts
- 3:19Riemann index and events
- 6:32Riemann streams explained
- 9:23Stream operations in Riemann
- 11:10Riemann configuration benefits
- 13:38Testing Riemann configurations
- 15:49Handling service downtime alerts
- 18:43Scaling Riemann instances
- 21:56Batching and async processing
- 22:42Q&A session
related talks
46mClojure Scaling the Event Stream
41mActivity Stream Processing in Clojure
42mBuilding a Data Pipeline with Clojure and Kafka
35mKafka and the REPL: Stream Processing, the Functional Way
31mOne Million Clicks per Minute with Kafka and Clojure
31mClojure at 4,000 msg/s-What We Learned, Loved, and Loathed
37mClojure Remixed: Pyroclast Data Processing
43mClojure in live sports television