Zeno: Diagnosing Performance Problems with Temporal Provenance
summary
Introduces temporal provenance and an algorithm that adds sequencing edges to provenance graphs to expose off‑path causes that only affect latency, not functionality. Implements the idea in the Zeno debugger, annotating nodes with precise potential speed‑up values so each event’s contributed delay is quantified. Evaluated by reproducing seven Google Cloud incidents, Zeno finds hidden maintenance jobs that queued RPCs to storage, shrinks graphs by ~89% and answers queries in under 10 seconds.
AI-written summary · about
chapters (20)
- 0:00Intro
- 0:27Motivation: Timing-related faults
- 1:13Can existing tools handle timing-related faults?
- 2:50Insight: Temporal causality is missing!
- 4:17How does temporal causality help?
- 5:08Approach: Temporal provenance
- 5:51Background: Provenance
- 6:56Challenge: How to capture temporal causality?
- 8:32Sequencing edges
- 9:24Challenge: How to read the graph?
- 9:55Annotating provenance with delay contributions
- 11:41Delay annotations: How to compute?
- 14:17Delay annotations: Formal Proof of Properties
- 14:32Provenance aggregation
- 14:53Implementation and experimental setup
- 15:22Evaluation: Questions
- 15:33What does the provenance look like?
- 16:34How readable is the provenance?
- 17:06How much time do diagnostic queries take?
- 17:24Summary . Problem: Diagnosing timing-related faults
related talks
15mRelational Debugging --- Pinpointing Root Causes of Performance Problems
38mTechniques and Tools for a Coherent Discussion about Performance in Complex...
46mHow to Properly Blame Things for Causing Latency
23mZen: Pinterest's Graph Storage Service
51mNavigating Distributed Traces
01h 03mZenoh: The Edge Computing Data Fabric
20mPerformal: Formal Verification of Latency Properties for Distributed Systems
23m30 Minutes To Zero: Slashing Debug Time With WhatIsWrongWithThis?