Scheduler Tracing With ftrace + eBPF
summary
Shows how to collect scheduler events (sched_waking, sched_switch, sched_wakeup_new) by enabling event files in tracefs and reading per‑CPU lockfree ring buffers, and visualizes them with perfetto trace viewer. Replays a pinned 3‑task workload (10,000µs/4,000µs, 5,000µs/2,000µs, 600µs/300µs) totaling ~65% CPU and observes a 3.67ms wake‑to‑run delay. Explains CFS internals (per‑CPU red‑black runqueue keyed by vruntime) and identifies the overload-on-wakeup scheduling bug; shows using ftrace + eBPF to reproduce and inspect behavior and mentions EEVDF as a newer scheduler to watch.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 1:09Why scheduler tracing matters
- 4:40Common server threading patterns
- 9:15Tracing scheduling events
- 13:56Analyzing scheduling latency
- 15:59Overload on wakeup bug
- 16:42Real-world impact of bugs
- 17:03Key takeaways and tools
related talks
55mAdvanced Tracing Features to See What Your Application is Waiting On
21mGraphing tools for scheduler tracing
30mScheduling at Scale: eBPF Schedulers with Sched_ext
35mOpening the Box: Diagnosing Operating-System Task-Scheduler...
59mProposing a new tracer to monitor RT task behavior
52mEffectively Measure and Reduce Kernel Latencies for Real-time Constraints
52mTracking Down What Happened: A Comparison of Tracing Solutions
28mLevel Up Your Game: OS Kernel and Game Interactions revealed with Perfetto