Build a Recommender System in Apache Spark and Integrate It Using Akka

on YouTube ↗48m

" summary

Shows how to train an ALS matrix‑factorization model in Apache Spark with specific hyperparameters (rank, iterations, lambda) and expose it as a REST service using Akka HTTP and actor-based background training so the API keeps serving an optional model while a new one trains. Validation uses logged recommendations plus click records (stealth testing) to compute precision/recall; target is roughly 80% precision, continuous retraining and tuning is recommended, and working example code is on GitHub.

AI-written summary · about

" chapters (7)

1. 0:00Start
2. 10:39Apache Spark Basics
3. 20:26Building the recommender system
4. 22:14Integrating everything into a micro service
5. 28:24When do the recommendations work?
6. 28:51Three easy steps
7. 32:17The number crunching

" related talks

• 31mCreating recommender systems with Spark, Scala and Prediction.IoFlorian Krause · code.talks 2017
• 40mIntroduction to Machine LearningJames Ward · Scale By the Bay 2017
• 29mThe Challenges of Building a Personalized Recommendation System in E-CommerceP. Poznic · Data Science Conference 4.0
• 26mLarge-Scale Recommendation System with Python and SparkPhil Anderson · PyOhio 2018
• 40mBuilding your own Distributed Streaming Recommendation EngineRui Vieira · Berlin Buzzwords 2018
• 31mNLP text recommender system journey to automated training pipeline with Spark and SagemakerAditya Sakhuja · Scale By the Bay 2020
• 01h 48mRecommendation engines explainedSophie Watson, Rui Vieira · DevConf.CZ 2019
• 45mReal-time, Streaming Advanced Analytics....Chris Fregly · Data By the Bay 2016