Creating recommender systems with Spark, Scala and Prediction.Io

on YouTube ↗31m

" summary

Concrete recipe: build recommenders with Spark Scala Prediction.IO, compute product similarity with cosine similarity (example result 0.981 = 98% similar), and train collaborative models with alternating least squares on RDDs reduced to 20–30 latent dimensions. Feature engineering includes one‑hot encoding and bucketing; events feed an event server (REST API, HBase/ES/MySQL backends), engines persist models to HDFS, serve via Akka with rolling updates, and run automated evaluation to tune parameters. Prediction.IO is maintained again by actionml after a dormant period.

AI-written summary · about

" chapters (14)

1. 0:00Start
2. 0:37Product Similarity
3. 0:58Scala and Functional Programming
4. 2:59Cosine Similarity Formula
5. 6:07Feature Engineering
6. 9:41Scala
7. 11:00Functional Programming
8. 14:37Spark Is a Database
9. 18:42Prediction Io
10. 19:05Event Server
11. 23:20Data Processor
12. 24:06Engine Dot Json File
13. 24:50Interfaces That Prediction Io Implements
14. 27:10Evaluation

" related talks

• 37mBuild a Recommender System in Apache Spark and Integrate it Using AkkaWillem Meints · Scala Days Berlin 2016
• 48mBuild a Recommender System in Apache Spark and Integrate It Using AkkaWillem Meints · Scala Days Berlin 2016
• 40mIntroduction to Machine LearningJames Ward · Scale By the Bay 2017
• 39mA recommendation engine for your applicationsMichele Orselli · Codemotion Amsterdam 2017
• 26mLarge-Scale Recommendation System with Python and SparkPhil Anderson · PyOhio 2018
• 43mBuilding your first recommenderJettro Coenradie · Codemotion Amsterdam 2018
• 32mLeveraging recommender systems to personalise search resultsSoraya Hausl · PyData London 2017
• 28mThe Power of Personalized Customer Experiences Through AI InnovationsN. Trajkov · DSC Europe 23