Cyclictest latency, find out why with tracing
summary
Runs cyclictest with kernel tracing enabled and uses Trace command plus new C libraries (libtracecmd, built on libtracefs and libtraceevent) to parse trace.dat, walk events backward to find the trace marker, then forward to correlate timer expiry, wakeup, sched_switch and locks. Claims: a 100 μs threshold hit as 561 μs in one run; function tracing and enabling all events raises average wake latency from ~3 μs to ~8–10 μs; a raw spinlock holds for ~168 μs. The tool automates analysis and mostly works; reverse iterator lives in Patchwork pending upstream.
AI-written summary · about
chapters (10)
- 0:00Cyclic test latency tracing
- 3:53Vanilla Linux kernel test
- 6:22Trace CMD for tracing
- 11:38Automating lib Trace CMD
- 13:23Lib Trace CMD code walkthrough
- 23:09Tracing software interrupts
- 31:39Tracing all latency events
- 36:16Q&A: lock owners and Python
- 40:10Q&A: upstreaming code and rtla
- 44:29Q&A: latency and overhead
related talks
49mAdding Inter-event Capabilities to Linux Kernel Trace Events
55mAdvanced Tracing Features to See What Your Application is Waiting On
41mSynthetic Events and Tracing Latency Within the Kernel
05mPerformance analysis using Systrace
42mHow Much Is Tracing? Measuring the Overhead Caused by the Tracing Infrastructure
49mDebugging Methodologies for Realtime Issues in Linux Systems
30mHow to get realistic c-state latency and residency?
52mTracking Down What Happened: A Comparison of Tracing Solutions