Scalable Anomaly Detection (with Zero Machine Learning)
summary
Claim: a real-time anomaly detector built on Zuul event streams and Mantis scales to thousands of nodes and handles millions of requests per second while using no machine learning. It implements a cheap median estimation / MAD-like algorithm (the “kodi” rule), Huffing bounds for recovery, and a rules engine that correlates Zul API Spinnaker and PlayReady signals. Outcome is concrete: targeted emails and timelines that pinpoint roots and reduced pages from ~10/week to ~1/week.
AI-written summary · about
chapters (42)
- 0:00Introduction
- 0:23Netflixs Microservices
- 3:36Time Series Database
- 4:01Alerts Only for Zul
- 4:44Dynamic Thresholds
- 5:03Adaptive Thresholds
- 5:28Anomaly Detector Raju
- 5:51Results
- 6:43No Machine Learning
- 6:57How We Built It
- 10:00Impact Graph
- 10:52Context
- 11:38Accuracy
- 11:58Operational Burden
- 12:23Realtime alerting
- 12:39Realtime events
- 12:49Mantis
- 13:42How it works
- 14:23Querying
- 14:53Stream Processing
- 15:08Aggregate
- 15:24Job Chain
- 15:49Requirements
- 16:27Median estimation
- 16:50Mad
- 17:29Raju
- 18:44Simple
- 19:07Recovery Detection
- 21:50Recovery Algorithm
- 23:00What Raju Looks Like
- 24:57Permutations
- 25:09Data Visualization
- 26:21Impact Assessment
- 27:52Timeline of Events
- 31:03What is API
- 32:15Another needle in a haystack
- 32:45Example
- 35:19Gold Standard KPI
- 37:07Spinnaker Events
- 37:35Emailing the culprits
- 39:17Benefits
- 40:13Conclusion
related talks
43mBeyond Logs: Real-Time Anomaly Detection with Machine Learning
34mML-Bound Anomaly Detection: Experiences Beyond Commodity Observability
29mREAL-TIME ANOMALY DETECTION MADE EASY
27mBeyond Logs: Real Time Anomaly Detection with Machine Learning
38mWatch Everything, Watch Anything: Anomaly Detection
34mAnomaly Detection in Time Series from Scratch Using...
26mProactive Failure Prediction Through Anomaly Detection
40mFraud detection without labeled data