Distributed In-Database Machine Leaning
summary
Apache Madlib brings machine learning into the database and runs in-database on Greenplum, HAWQ and Postgres. Algorithms are implemented in C++ using Eigen and Boost and execute with a distributed iteration loop: transition, merge, final on MPP segments. Demos show linear regression with per-group models, one-class SVM for novelty/fraud detection, and a path/sessionization utility. Performance scales roughly linearly to hundreds of millions of rows; roadmap adds mini-batching, graph algorithms, GPUs/TensorFlow and gradient-boosted trees.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:53Machine learning is eating the world
- 1:11Google Trends
- 1:47Interest in Machine Learning
- 2:30Relational Data
- 4:17What is Madlib
- 6:04Madlib functions
- 15:03History of Madlib
- 17:21Academic Partners
- 18:14Supported Platforms
- 18:47Greenplum Architecture
- 21:00Query Optimizer
- 22:06Other Features
- 23:19Algorithm Scalability
- 28:03Linear Regression
- 30:34Execution Flow
- 31:44Stored Procedure
- 35:13Demo
- 38:40Support Vector Machines
- 41:00Training Data
- 41:48Decision Boundary
- 42:57Outlier Data
- 43:56Path Function
- 46:12Upcoming Work
- 48:36Questions
related talks
35mBreakout Session
33mScalable in-database machine learning with PL/Python
34mA machine learning hub for IoT data using Apache Madlib
40mIn-Database Machine Learning with Jupyter
25mPostgres and the Artificial Intelligence Landscape
31mIn-database machine learning: Automate, scale, deploy
41mMassively Parallel Processing with Procedural Python
28mCharge Your Database with Predictive ML Capabilities using Open-Source