Hot Deploying Low-Latency Services for 24/7 Operation
summary
Shows how to run 24/7 low‑latency services with hot deploys by using replicated state machines + consensus (Raft), snapshots + log replay to bound recovery, and asynchronous multicast replication to avoid RPC timeouts. Claims systems reach sub‑20 microsecond latency at over a million events per second while tolerating node upgrades by rolling one node at a time and using 5‑node clusters to tolerate concurrent upgrades/failures. Recommends versioned messages, bitmask feature flags, idempotent operations, and Aeron as a proven transport.
AI-written summary · about
chapters (10)
- 0:00Intro
- 6:4824/7 Operations
- 8:57High-availability
- 13:14Replicated state machines
- 19:47Rolling upgrades
- 25:24What about my services?
- 36:16Disaster recovery
- 40:11What helps at extremes?
- 42:55Aeron
- 43:19Outro
related talks
13mLow-Maintenance Backend Architectures for Scalable Applications
18mScaling APIs to Billions: Extreme-Scale Reliability
47mDeploying Like a Boss: From 2 Servers to 20,000
48mFault Tolerance on the Cheap Making Systems That Probably Won't Fall Over
18mScaling, what you need to know
45m⚡A Resilient Application in a Partially Degraded World ⚡
40mStrategies for Zero Down Time, Frequent Deployments
27mNomad Best Practices for Reliable Deploys