Resurrecting a Recommendations Platform
summary
Built a GPU-backed deep learning recommender while migrating a bare‑metal platform to AWS, keeping millions served. Replaces an 18‑hour batch transform with an event-driven Kinesis→Lambda→S3 ETL and preserves raw Avro for reprocessing. Orchestration moves from ad hoc scripts to Airflow orchestration, training daily and auto‑retraining on distribution shifts. Adds online evaluation via a/b testing tier and fast feature access through Redshift and DynamoDB snapshots, and measurably reduces staleness and scale failures.
AI-written summary · about
chapters (5)
- 0:00Start
- 3:40What does it mean to resurrect a recommendations
- 5:23If you compare machine learning to your standard software development for some reason the best practices are not applied.
- 11:08Orchestration of Machine Learning - If we made a change how long would it take to
- 30:05Monitoring & Performance Metrics
related talks
33mPowering Real-time Recommendations with Machine Learning on GCP
14mBattle-tested tactics to accelerate data science and ML R&D with AWS
38mRecommendations at Rappi: from MVP to Data Product
21mAWS On Air ft. Amazon SageMaker Inference Recommender
19mX1 Recommendations Platform: Started from the bottom..
48mEnd to End Collaborative Data Workflows
41mRecommendations for Building Machine Learning Software
27mEnhancing Personalization Using Recommendation Systems & Multi-Armed Bandits