Massive Scale Anomaly Detection Framework
summary
Describes PayPal’s pipeline-based, scikit-learn/Spark MLlib–inspired framework built on Spark Structured Streaming for real-time, scalable anomaly detection. Covers designing a generic, flexible API and implementation patterns to adapt traditional batch analytical models to unbounded streaming datasets and low-latency requirements.
AI-written summary · about
chapters (4)
- 0:00Start
- 6:48What is the current observation?
- 11:43Traditionally statistical and ML models have been designed for batch processing
- 25:41Bound and reduce processing time
related talks
18mReal-time Fraud Detection using Deep Learning
44mAnomaly Detection in Telecom with Spark
43mBeyond Logs: Real-Time Anomaly Detection with Machine Learning
52mDevil in the Haystack
19mThe Rise of AI in Anomaly Detection
20mSmart Monitoring System for Anomaly Detection on Business Trends
29mAI-Powered Anomaly Detection in Cloud Data Pipelines
39mMastering real-time anomaly detection with open source tools