DataEngBytes 2023 - SYD-T3-03 - Suzanne Nieuwenhuizen
summary
Shows a production pattern that separates stable entities and high-velocity events with Neo4j Fabric, using shards for person/route map/waypoint/asset/container data and a federated layer to join them. One deployment holds 19 billion nodes in the person shard and 1.2 trillion relationships in the forum/container shard while keeping read/write latency under 20 ms. Uses Neo4j 5 autonomous clustering, Kafka/Spark connectors, and different instance types to balance cost, retention and real-time ingestion.
AI-written summary · about
chapters (6)
- 0:00Introduction
- 1:45Graph databases prevent row locking
- 5:39Graph vs relational and document databases
- 9:31Scaling knowledge graphs with sharding
- 15:59Real-world shipping route network
- 25:30Key takeaways and closing remarks
related talks
26mHow NEO4J Saved my Relationship
23mDisrupting Customer Data Platforms: Neo4j vs. the Status Quo
30mBeyond the Silos Achieving Data Harmony With Graph Based ER
29mTraversing from Zero to Production with Graphs
28mNeo4j for Beginners: Pros and Cons, Based on Social Networks
30mContext Engineering: Connecting the Dots with Graphs
24mOur daily graphs written in Ruby and Neo4j
27mNeo4j GDS for Snowflake