Real Time Sensor Anomaly Detection with Sci Kit Learn and the Azure Stack
summary
Real-time pipeline detects and separates sensor faults from real inflow/infiltration using Azure Machine Learning, scikit-learn (random forest and a hybrid classifier over outlier picks), Event Hubs, Stream Analytics and Power BI, with a prototype dataset showing 495 labeled anomalies. Models train in notebooks and deploy as Azure ML web services (production endpoint currently Python 2.7), devices post streams to Event Hubs for scalable ingestion and Stream Analytics for one-second processing and visualization.
AI-written summary · about
chapters (16)
- 0:00Intro
- 0:44About me
- 1:07Municipal wastewater management
- 1:26Why do we care
- 3:22Traditional methods
- 8:34Code
- 9:11Machine Learning Studio
- 10:24Notebook Service
- 12:10Dataset
- 13:13Binary Classification
- 15:17Playing with Data
- 16:15Visualizing Data
- 18:28Comparing Models
- 19:19Building Models
- 22:01Azure ML Client API
- 23:24Testing
related talks
29mAI-Powered Anomaly Detection in Cloud Data Pipelines
37mPracticing Data Science in the cloud
19mEnterprise AI-Powered Predictive Maintenance Platform
39mDeveloping Spidey Senses
01h 01mStreams, lakes and oceans – working with on big data with Azure ML
26mFrom Data to Decision: Monitoring and Improving Water Networks using AI/ML
51mAnomaly Detection as a Service - What is normal?
41mPredictive Maintenance with Azure Synapse