Large Road-Network analysis with GraphX
summary
Uses GraphX on Spark to turn OpenStreetMap ways into a dual road graph (sections between intersections), processes ~80 cities totaling ~3 million section-edges (Los Angeles ≈200k, smallest ≈200). Runs PageRank and normalizes by city size, compares histograms with an information‑theory divergence to cluster topologies. Finds PageRank plus connectivity features correlate with car‑accident risk; a disease‑spread attempt fails. Prototype scales on a four‑node m3.large AWS cluster and runs a few PageRank iterations on city graphs in seconds.
AI-written summary · about
chapters (11)
- 0:00Road-network analysis intro
- 5:45What is GraphX
- 11:09PageRank and Pregel
- 18:18PageRank on city streets
- 23:16Normalizing PageRank
- 26:46Measuring topology divergence
- 29:19When to use GraphX
- 32:35Scaling GraphX globally
- 35:27Real-world applications
- 40:08GraphX optimizations
- 41:15Q&A
related talks
38mGraphs Are Everywhere!
01h 41mGraph Intelligence: Enhance Reasoning and Retrieval Using Graph Analytics
15mSee What the Graph Sees: Demystifying PageRank Through Visual Storytelling
01h 38mNetwork Science for Fun and Profit
37mWorking with Maps: Extracting Features for Traffic Crash...
40mRun Large-Scale Graph Analytics Using The Most Popular Graph Analytics Library Available
46mGraph Algorithms: Predict Real-World Behavior
32mUsing a Graph Database Tree and Data Analytics to Model an Ancient River System