Scalable in-database machine learning with PL/Python

on YouTube ↗33m

" summary

Shows how to run scikit learn models inside Postgres via PL/Python using a wine dataset to train a ridge regression on wine dataset and returns coefficients, intercept and R². Demonstrates the 1 GB field limit for UDF inputs and a practical workaround using a global GD cache to stream rows into Python so models can use tens of GB of server RAM. Presents Apache MADlib as the C++-backed option for model-parallel, elastic net and other algorithms that run faster than plain scikit learn on Postgres/Greenplum/Hawq.

AI-written summary · about

" chapters (12)

1. 0:00Start
2. 6:08Model Parallel Problems
3. 7:31Wine Quality Data Set
4. 9:44Example of Building a Machine Learning Model
5. 10:04Helper Functions
6. 10:47Model Coefficients
7. 11:17User-Defined Aggregates
8. 12:30Pl Python Function
9. 19:12User Defined Aggregate
10. 24:22Building Scalable Models
11. 24:55Machine Learning Library
12. 30:11The Predict Function in Matlab

" related talks

• 55mDistributed In-Database Machine LeaningFrank McQuillan · PGConf Silicon Valley 2016
• 25mPostgres and the Artificial Intelligence LandscapeBruce Momjian · Citus Con: An Event for Postgres 2023
• 01h 00mPostgreSQL and Artificial intelligenceIbrar Ahmed · Percona Community Live 2022
• 45mPostgres and the Artificial IntelligenceIbrar Ahmad · Percona Live 2022
• 35mBreakout SessionAJ Welch · PGConf Silicon Valley 2015
• 41mMassively Parallel Processing with Procedural PythonRonert Obst · PyData Berlin 2014
• 49mPostgres and the Artificial Intelligence LandscapeBruce Momjian · PGConf.dev 2024
• 49mPostgreSQL - the database-sister of PythonHarald Armin Massa · EuroPython 2011