Large Scale CTR Prediction Lessons Learned
summary
Yelp runs logistic regression with thousands of features and millions of samples and reports that model training on Spark via MRJob/AWS EMR completes on the order of minutes. Production practices include log at source and prediction verification to keep offline training aligned with online behavior, automated fast offline iterations, and evaluation driven by mean cross entropy. Thresholding the PCTR creates feedback bias; deliberate sampling of low‑PCTR impressions and combined scoring for A/B comparisons recovers hidden performance near cutoffs.
AI-written summary · about
chapters (11)
- 0:00CTR prediction at Yelp
- 1:06Why Yelp’s CTR matters
- 4:28CTR prediction architecture
- 6:17Logging infrastructure lessons
- 8:45Fast offline iterations
- 12:20Model evaluation
- 13:02Single evaluation metric
- 17:17Threshold effects in CTR
- 21:37Combined scoring for bias
- 24:05Simplicity in ML systems
- 26:14Lessons learned
related talks
30mPredictive Analytics in Social Media and Online Display Advertising
27mLarge Scale Response Prediction to Serve Ads on LinkedIn
37mSupercharging CTR (Click-Through Rate) with Plug-and-Play AI Capabilities
30mMobile Order Click-Through Rate (CTR) Recommendation with Ray on Apache Spark at Burger King
53mReal-Time Big Data Analytics with R
29mMachine Learning in Production: What to do after you’ve trained the models?
44mMachine learning @ booking.com
38mUsing Recurrent Neuronal Networks for Customer Predictions