Building an recommendation system for IPTV
summary
A production IPTV recommendation pipeline built on a fast streaming stack uses Kafka for ingestion, Vertica for storage, Spark (PySpark) with ALS matrix factorization for collaborative filtering, and a daily job that writes top-10 personalized lists back into Vertica; implicit ratings map percent-watched to a 2–5 scale. Deployed on several vendor platforms, it handles a few hundred thousand users of clickstream data, rejected Cassandra for write bottlenecks, and supports both real-time and batch queries. Kafka ingestion, Spark ALS on implicit feedback, Vertica storage and daily top‑10 recompute
AI-written summary · about
chapters (16)
- 0:00Start
- 4:58Introduction
- 10:03Streaming services Using Internet and IP protocol for distribution of the content
- 10:46IPTV vs. OTT
- 12:39Business requirements
- 13:36Architecture of the solution
- 15:22Technical solution - Flexible data ingestion
- 15:53Technical solution - Kafka integration feature
- 16:40Business Solution
- 18:16Recommender systems for content delivery
- 19:11Approaches
- 22:06Recommender system as a part of Pl Content Analytics system
- 23:29Chosen approach
- 24:10Tools and techniques
- 25:16Implicit ratings
- 26:48Conclusion
related talks
40mBuilding your own Distributed Streaming Recommendation Engine
19mX1 Recommendations Platform: Started from the bottom..
26mLarge-Scale Recommendation System with Python and Spark
20mDataEngBytes 2023 - PER-T1-06 - Kamal Hossain
04mInterview
30mFive Challenges in Real-time Stream Processing and Five Solutions
42mBuilding Robust Streaming Data Pipelines with Apache Spark
31mNLP text recommender system journey to automated training pipeline with Spark and Sagemaker