5 Simple Steps to Create Meaningful Features from Clickstream Data
summary
Uses Mixpanel logs (~10 GB) parsed with pandas (considering Dask for scale) and sessionized at a 30‑minute gap to build median session duration and shortest‑session page count plus URL visit binaries and short sequences; experiments include Markov chains and LSTMs (too little data), final model is XGBoost with LIME for explanations. Prevents leakage by using features within the first 24 hours. Initial model hits ~78% ROC with leakage, drops to ~68% ROC after removing leakage‑reflecting features, which aligns with industry baselines.
AI-written summary · about
chapters (9)
- 0:00Introduction to project
- 3:04Project workflow overview
- 3:57Data acquisition and goals
- 6:15Methodology exploration
- 9:53Data pre-processing
- 13:43Session characterization
- 17:17Feature engineering
- 23:08Handling data leakage
- 25:38Model results and conclusion
related talks
34mHow to understand user behaviour using Multiple Linear Regression?
49mUnderstanding Customer Journeys with Deep Learning
02h 53mMining digital exhaust
53mReal-Time Big Data Analytics with R
11mData science: Next-gen performance analytics
36mML: Predictive analytics in Ad Engineering
40mApplying Data Science and Analytics in Marketing
25mHow we bring Performance Marketing to linear TV by using machine learning