From Microbenchmarks to HTTP2 Load-testing: 5 Performance To...

on YouTube ↗26m

" summary

Envoy startup has a 20x startup speed-up after fixing quadratic behavior (from >10 minutes to 22s) and then reworking regex checks to reach 3.75s; switching regexp library and rewriting a hot pattern cuts per-match cost from ~1.3µs to <0.5µs and yields a ~10% overall start improvement. A new Nighthawk load generator (Envoy networking stack) drives HTTP/2, offers closed/open loop modes, reveals a 5µs HTTP/1 vs HTTP/2 gap and a 2ms long tail traced to CPU pinning reduced to 600µs. Instrumentation via regex microbenchmarks and fuzzer found N² bug (CVE) round out the toolset.

AI-written summary · about

" chapters (28)

1. 0:00Intro
2. 0:18Adding Observability to a Service Oriented Architecture
3. 1:16Scaling Dimensions
4. 2:04Situational use of tools and techniques
5. 2:15Case-study: stat scalability issues during startup
6. 3:00Stat scalability phase 2: regex tag init bottleneck
7. 4:11Performance Annotations in stats tag extraction
8. 5:37Prefiltering to reduce the number of regex evals
9. 7:22Next steps toward improving performance
10. 9:07Microbenchmarks comparing variants
11. 10:34Take-aways: Microbenchmarks and Annotation
12. 12:22Load Generation Evaluation funnel
13. 12:35Detailed Evaluation: WRK2
14. 12:50Detailed Evaluation: fortio
15. 13:22Decision: build a load-generator on top of Envoy
16. 14:33Nighthawk: closed-loop mode
17. 14:53Nighthawk: command line closed-loop test
18. 15:08Nighthawk: open-loop mode
19. 15:30Nighthawk: command line open-loop test
20. 15:44Nighthawk: zooming in
21. 16:07Nighthawk: single threaded pinned test server
22. 16:17Nighthawk: Measuring small differences
23. 17:33Nighthawk: Envoy H1 vs H2 performance
24. 18:11Nighthawk: tracing
25. 18:35Nighthawk: notable features
26. 19:12Finding performance problems with fuzzing
27. 19:31Finding Performance Problems from generated patterns
28. 21:47Fuzzing performance challenges

" related talks

• 39mLast-Minute Performance TestingPavlo Vedilin · QA Fest 2018
• 36mPerformance Optimization 101Louis-Philippe Gauthier · Erlang Factory SF Bay Area 2014
• 56mBetting on Performance: A note on Hypothesis-driven Performance TestingJames Lewis · NDC Oslo 2017
• 21mBenchmarking Like a BossJohn-David Dalton · JSConf US 2011
• 27mEffective Performance Testing to your Server with AutocannonTamar Twena-Stern · TestJS Summit 2021
• 38mYour benchmark may not guide a real application performanceTetsuharu OHZEKI · JSConf JP 2019
• 02h 44mMeasure, don’t guess: benchmarking stories from the trenchesMario Fusco, Francesco Nigro · Devoxx Belgium 2024
• 39mMicro BenchmarksStoyan Stefanov · performance.now() 2022