A machine learning hub for IoT data using Apache Madlib
summary
Sets up a local IoT hub that keeps sensor CSVs as virtual tables in Postgres and runs in-db ML with Apache Madlib using the file_fdw foreign data wrapper and extraction via QS Ledger on Oura/Apple CSVs. Demonstrates end-to-end workflow on ~300 MB compressed (≈900 MB uncompressed) of personal sleep/HR data, creates views with computed day columns, and finds a notable correlation (weak 0.09 for sleep score but moderate–strong between deep sleep and next-day RMSSD), working but exploratory.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 0:52Project overview
- 4:46Nitins background
- 6:29Backstory
- 9:02Why an IoT hub
- 10:45Assumptions
- 11:42Apache Madlib
- 13:20Tools
- 15:49Setup
- 19:31Steps
- 20:55Data Wrapper
- 21:29Data Types
- 22:46API Integration
- 23:41Apple Health
- 26:32Demo
related talks
55mDistributed In-Database Machine Leaning
01h 26mData Driven Action: a primer on Data Science
29mML-Powered IoT: From Warehouse Data to Prod Intelligence in Real Time
42mSo, You Want to "Do Big Data?"
35mBreakout Session
52mMachine Learning in IoT Applications
31mFun with Sensors
37mFrom Sensors to Insights at the Edge: Applying AI to IoT Data