Tracing the LLM Behaviour: Debugging and Evolving Agent Workflows with Neo4j
summary
A practical pipeline captures every agent event into a Neo4j graph of steps, tools, memories and outputs via a lightweight observer that emits Cypher inserts (demo uses Docker Compose and a Streamlit UI). The live demo shows a failed run with 3 nodes then a successful run with 5 nodes, proving replay and repair work end-to-end. Built-in GDS algorithms find shortest paths and failure clusters, and developers report debugging time dropping from hours to minutes using Cypher queries to trace failures.
AI-written summary · about
chapters (10)
- 0:00Introduction
- 1:05Problem with LLM workflows
- 4:53Proposed solution with Neo4j
- 7:05High-level architecture
- 10:33Graph model components
- 14:27Demo setup and execution
- 18:08Debugging workflow failure
- 21:44Neo4j GDS algorithms
- 24:12Advantages of Neo4j
- 26:20Q&A session
related talks
16mGraph Traversal Agent for Code Debugging
01h 03mAgents Everywhere
56mHerding LLMs: Predictable Outcomes from Unpredictable Agents
22mBuilding Cypher Queries With Open-Source LLMs
32mMulti-Agent Query Generation w/ LLMs on Complex Knowledge Graphs
25mFrom Chaos to Control: Building Enterprise-Grade LLM Agents That Actually Work
30mNeurosymbolic Architecture in Production with Neo4j, GDS, Node2Vec & LLM Grounding
23mConnecting the Dots With Context Graphs