Making Envoy Resilient to Sudden Increases in Load
Talkat EnvoyCon NA 2024 · on Tuesday, 2024-11-12· at 09:10· Hyatt Regency | Level 2 | Salt Lake Ballroom C·
summary
Analysis of last year’s HTTP/2 Rapid Reset shows attackers reached ~400M RPS globally and caused huge CPU and memory bookkeeping churn, so Envoy adds concrete defenses: max requests per IO cycle to limit work per TCP segment, a proactive low setting that polls memory usage for faster shedding, and tuning of max connections to accept per socket event plus buffer limits. Benchmarks report reduced latency amplification and earlier overload action triggering, preventing memory exhaustion and enabling graceful connection goaway.
AI-written summary · about
chapters (14)
- 0:00Introduction
- 1:06DDoS attacks impact
- 2:34HTTP/2 Rapid Reset explained
- 4:41CPU and latency impact
- 6:56Sidecar failures and cascading effects
- 8:43Envoy mitigations for Rapid Reset
- 9:45Envoy’s overload manager
- 11:34Proactive load shedding
- 13:47Critical load-shedding points
- 15:37System resilience configs
- 17:04Max requests per IO cycle
- 18:10Max connections per socket event
- 18:54Buffer size best practices
- 19:58Conclusion
related talks
27mEnvoy - Overview and Update: Senior Envoy Maintainer
15mFail Open, Fail Fast: Improving Envoy Resilience in Latency-Critical Systems
14mKeeping Envoy Gateway On, Even When Bad Things Happen
29mImproving performance of RPCs with envoy
37mEnvoy Internals Deep Dive
10mLightning Talk: Envoy @ Netflix
25mScaling Up, One Network Bottleneck at a Time
25mBuilding Low Latency Topologies with Envoy