Estimating solar production w/ distributed deep learning from smart meter data
summary
Claim: estimate per‑interval household solar production from smart‑meter redelivery alone, using day‑level inputs of 96 per‑day 15‑minute intervals. Implementation uses PyTorch and Horovod for distributed training (data‑parallel, larger effective batch, higher LR to speed convergence), Spark for parallel daily predictions, and MlFlow for model/version registry; pipeline feeds the Eneco/Toon mobile app. They train on labeled thermostat/solar module data, production estimates look smooth and run at scale for daily delivery.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:11Smart meter solar measurement
- 4:00Smart meter data pipeline
- 6:30Model training flow
- 7:34Input data and labels
- 9:20Neural network architecture
- 11:25Distributed deep learning with Horovod
- 13:33Data parallelism and learning rate
- 16:20Horovod integration example
- 18:49Experiment tracking with MLflow
- 20:33Model versioning and storage
- 22:22Scaling predictions with Spark
- 23:45GPU vs CPU costs
- 25:29Closing
- 25:51Q&A
related talks
40mDeep Learning and Time Series Forecasting for Smarter Energy
11mUsing AI to Accelerate the Smart Green Transition
27mUsing AI for cheaper and more efficient solar power
23mDataEngBytes 2023 - MEL-T3-01 - Rajath Vanikul
30mOutsmarting the energy grid with AI
45mWorkshop: Deploy and Monitor ML Pipelines with Open Source and Free Applications
01h 16mPredicting Weather Patterns using Time Series ML
42mPredicting Weather-Caused Rare Events: A Utility Outage Prediction Use Case