Stateful Connections in Kubernetes: The Scaling Secrets Nobody Talks About
Talkat KubeCon + CloudNativeCon Europe 2025 · on Friday, 2025-04-04· at 15:15· Level 1 | Hall Entrance N10 | Room H·
summary
Miro rebuilds its WebSocket routing by deploying a custom WebSocket manager on EKS, fronted by AWS ALB + target group binding, replacing Fabio and console; pods handle about ~8,000 connections per pod (peaks higher) while using connection-aware autoscaling, graceful drain hooks and split readiness endpoints to avoid dropping live sessions. Results show initial connection latency falls 10×, cold-start overloads are prevented, and infrastructure cost drops by roughly $40k per year.
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:05Miro’s product history
- 5:18Legacy architecture and Kubernetes
- 8:52Stateful connections fundamentals
- 14:36New websocket manager design
- 17:25Graceful shutdown and load balancing
- 19:11Horizontal autoscaling for connections
- 21:19Load balancing algorithms and pitfalls
- 25:10HPA challenges and policies
- 28:41Results and cost savings
- 30:06Future steps and closing
related talks
32mScaling Kubernetes Application to 1M Users
35mKubernetes, Lessons Learnt
39mService Mesh: There and Back Again
29mWhy We Ditched Kube-proxy: Scaling 10M Daily Browser Sessions With Kubernetes Endpoi...
27mRails Orbiting the cloud: Mastering the scalability with Kubernetes
34mLatest Kubernetes Scalability Improvements
30mNetwork Configurations for High-Performance Streaming Connectivity in K8s
34mEmbracing Upstream Kubernetes in Web Scale Organization