Pythia: An Automated, Cross-layer Instrumentation Framework for Diagnosing Performanc...
summary
Pythia is an automated cross‑layer instrumentation framework that explores which trace points to enable using distributed tracing plus statistical search, implemented by augmenting OSProfiler on OpenStack. It represents instrumentation choices with calling context trees and uses hierarchical search and CCA to pick measures; a reproduced case spots a semaphore cap that limits VM builds to 10, explaining high latency variance. One experiment shows 95% variance captured by six trace points out of ~350, and a queue‑length variable has 0.85 correlation with request latency.
AI-written summary · about
chapters (13)
- 0:00Start
- 7:41Pythia Architecture
- 8:32Instrumentation Plane Requirements
- 9:53Search Space
- 10:49Caling Context Trees
- 12:08Hierarchical Search
- 12:53Other Search Strategies
- 15:35Highest Variance Edge
- 16:12Finding self. build semaphore
- 16:45Adding More Instrumentation
- 17:12Canonical Correlation Analysis (CCA)
- 18:04After Enabling Instrumentation
- 18:57What about Other Trace Paints?
related talks
01h 00mSystems @Scale Summer Q&A Week 4
38mTechniques and Tools for a Coherent Discussion about Performance in Complex...
41mGain a deeper understanding of your incidents: continuous profiling
01h 01mThe Performance Investigator's Field Guide
24mCommunity Driven Breakout Session: Continuous Profiling
26mA Year of Observability Across the Toolstack: Breakthroughs, Challenges, Improveme...
44mThink Outside the VM: Unobtrusive Performance Measurement
36mContinuous Profiling with Parca- Beneath the Kernel to Beyond...