Go Profiling and Observability from Scratch
summary
Simple Go runtime model plus hands-on tools: CPU, memory, block, mutex, goroutine profilers and the runtime tracer, plus eBPF (bpftrace), Linux perf, Delve and a small ftprof prototype. Claims: heap allocations and GC can cost ~20% CPU; CPU profiler samples at 100Hz (setcpuprofilerate), Go 1.18 fixes a 250-signal limiter; block profiler and tracer can add large overhead on heavy channel/mutex workloads while cpu/mem/mutex are usually safe in production. Concrete knobs: block rate ≈10000, mutex rate ≈10.
AI-written summary · about
chapters (32)
- 0:00Intro
- 0:15Target audience
- 0:26Agenda
- 1:26Scheduling & Memory Management
- 5:32Memory Management: Stack
- 6:59Memory Management: Heap
- 11:06CPU Profiler: SIGPROF for every 10ms of CPU Time
- 11:51CPU Profiler: Add Stacktrace to Profile
- 12:02CPU Profiler: Waiting for Go 1.18
- 13:28Block Profiler: Channels
- 14:19Block Profiler: Mutexes
- 14:52Mutex Profiler
- 15:17Block vs Mutex
- 16:34Quick Note on Time: 1 Request - 3 Goroutines
- 18:41Memory Profiling
- 20:32Goroutine Profiling
- 22:18Tracing Manually
- 24:33Runtime Tracer
- 26:18Profiling and Tracing: A Faustian Bargain?
- 26:50Profiling and Tracing Overhead Analysis
- 27:34Overhead Analysis: SQL Workload
- 29:03Overhead Analysis: HTTP Workload
- 30:02Overhead Analysis: JSON Workload
- 30:51JsoN Workload (Concurrency = 8)
- 33:47Overhead Analysis: Chan Workload
- 34:53Overhead Analysis: Error Sources
- 36:16Metrics
- 38:01Linux perf
- 38:36bpftrace
- 41:34Scheduling / Execution Observability
- 42:12Memory Management Observability
- 42:42Recap
related talks
33mBuilding a Go Profiler Using Go
40mTwo Go programs, three different profiling techniques, all in 40 minutes
38mFighting latency: the CPU profiler is not your ally
45mBoost application performance with profile guided optimization
48mProfiling is for Dev, Monitoring is for Ops – no more!
56mThe Key to Go Efficiency is Just a Few Go Runtime Metrics Away!
34mGolang: Specific Performance Issues
28mDeep Dive Into Profilers: How CPU Profilers Measure Your Application's Performance