Serious Network Analysis Using Hadoop & Neo4j
summary
Shows a concrete pipeline that turns raw logs into a Neo4j graph: Hadoop + Cascading to extract unique nodes and edges, Hive joins to merge feature aggregates, then a batch inserter to bulk-load the DB. Demonstrates on an Internet AS graph, uses MapReduce graph partitioning to keep partitions manageable, and a browser JS viewer that merges query results live. Claims a full import can finish in 30 minutes if IDs fit in memory, otherwise performance degrades due to swapping.
AI-written summary · about
chapters (6)
- 0:00Why networks matter
- 3:57Building network databases
- 7:58Hadoop for raw data processing
- 12:24Neo4j graph database import
- 16:00Querying and visualizing graphs
- 18:37Tools and resources summary
related talks
45mSolving Real & Big (Data) Problems Using Hadoop
28mInterview with
24mGraph-Powered Job Matcher: From SQL Tables to Neo4j RAG in Production
38mNeo4j for Solving Graph-Based Problems
24mOur daily graphs written in Ruby and Neo4j
28mNeo4j for Beginners: Pros and Cons, Based on Social Networks
25mBuilding an Entire Graph Application with Nodestream
29mCDC Analytics With Graph Stream Zoomer