Recommendation engines explained
summary
Uses the MovieLens 100k dataset and Jupyter/Binder to demonstrate Spark ALS collaborative filtering, trains with a grid over rank {6,8} and maxIter {10,12} (grid search runs ~1m6s on Binder), finds best rank 6 and maxIter 12, and improves mean squared error versus a naive mean baseline (baseline MSE ≈ 1.08). Shows that simple post-processing (drop movies with <100 ratings) makes top-N results sensible, describes a production pattern with OpenShift microservices (separate modeler and predictor) and an online bias-stochastic gradient update.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 6:40Jupiter Notebook
- 12:50Dataset
- 13:30Spark
- 16:23Load CSV
- 16:59Spark locally
- 17:30Count
- 18:15Lazy
- 19:55Rating
- 20:42Ratings
- 22:07Movies
- 23:06Recommendation Engines
- 23:32Collaborative Filtering
- 35:03Max iterations
- 36:29Tuning parameters
- 39:47Spark ALS
- 46:03Parameter estimation
- 49:46Maximum iterations
- 52:41Predictions
- 54:06Error distribution
related talks
15mWhat they didn’t teach you in Grad School #bbuzz
40mBuilding your own Distributed Streaming Recommendation Engine
01h 21mUnlocking Personalization: A Deep Dive into Modern Recommendation Algorithms
01h 31mHands on - Build a Recommender system
37mBuild a Recommender System in Apache Spark and Integrate it Using Akka
33mPicking What to Watch Next: Build a Recommendation System
37mA primer on recommendation systems.
46mBeyond Bag of Words A Practitioner’s Guide to Advanced NLP