From Polling to Pushing: Going Realtime with Go, gRPC, and Redis
summary
Propeller replaces frontend polling with a push model using bidirectional gRPC over persistent TCP connections, implemented in Go with goroutines and Redis (optional NATS) for Pub/Sub or Streams. It routes events via a broker to the container that holds a client connection, drops front-end P95 latency from 5s to 250ms P95, handles tens of thousands of concurrent users and about 300k goroutines on 3–4 containers (~3 GB), costs ≈ $200/month and is open source.
AI-written summary · about
chapters (9)
- 0:00Introduction and context
- 2:55Polling use cases
- 4:32Challenges of polling
- 5:49Introducing Propeller
- 7:38Technical design and gRPC
- 9:26Connection affinity and architecture
- 11:53High concurrency with Go
- 13:28Impact and open source announcement
- 14:39Q&A session
related talks
28mManaging and Scaling Real Time Data Pipelines using Go
18mPolling: It's Good Enough for the WWW & It's Good Enough for You
01h 16mGoing Async - Practical Patterns for Push-Enabled Applications
28mThe End of Polling
30mHigh Performance Systems in Go
36mPush it push it some more
45mBeyond REST: Using Full-Stack Signals for Real-Time Reactive UIs
31mBuilding a microservices architecture with Go