Mall Analytics Using Telco Data & Pandas
summary
Estimates mall visitor trends from telco event records using Hadoop/map‑reduce, HDFS, Avro and Python, producing CSVs for a dashboard and preserving privacy while correcting visibility biases. Produces visitor counts per hour and a map of home locations showing transit‑linked catchment areas; uses pandas matplotlib scikit‑learn and a t test that finds a significantly higher median visit duration for repeat visitors (very small p‑value). Workflow moves toward Luigi for reliable jobs.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:38What telco data looks like
- 3:40Own platform demo
- 7:25Architecture
- 9:25Live coding time
- 20:58Evaluation of results
- 22:11History about project evolution
- 24:57Using Luigi for Workflow monitoring
related talks
01h 52mIntroduction to data analytics with pandas
01h 31mPandas, Data Wrangling & Data Science
43mPython at City Scale
34mML and analytics addressing nationwide COVID-19 impact and recovery
34mTracking Policy Evolution Through Clustering
42mTackling Homelessness with Open Data
24mTalk Data to Me - the art of storytelling
40mData-centric exploration using intake, dask, hvplot, and more