Analysing Real time Activity Streams for Security
summary
Shows how Apache Kafka and Flink ML are combined to run machine‑learning models on cloud activity event streams to detect malicious access patterns in near real‑time. Describes applying heuristic and model‑based detection to cloud data collaboration platforms and considerations for scaling the pipeline to high‑throughput systems.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:30Qualys Platform Environment Security at scale on cloud
- 0:49Qualys Cloud Platform Highly performant fully distributed microservice based big data stack
- 1:16Microservices & Cloud Native Architecture
- 1:46Problem Statement : Threat detection from User Activities on Cloud Applications
- 2:37Possible Solutions
- 3:38Machine learning Approach For Threat Detection
- 4:02Flink: Overview
- 5:53Flink: Use case
- 7:11Flink: State Management
- 7:58Flink: Broadcast State
- 8:34Architecture: Detail
- 11:35Architecture: Model Training
- 12:47Architecture: Load and Apply Model
- 14:46Best Practice
- 15:45References
related talks
43mBeyond Logs: Real-Time Anomaly Detection with Machine Learning
25mReal-Time Fraud Detection at Scale
27mBeyond Logs: Real Time Anomaly Detection with Machine Learning
29mIntelligent Log Analysis and Real-time Anomaly Detection @ Salesforce
40mHow to leverage the Apache Kafka Ecosystem to productionize Machine Learning
51mReal-time Fraud Detection: Challenges and Solutions
37mBig Data Analytics, Machine Learning, Real Time Processing
47mReal-time Fraud Detection Challenges and Solutions