Deploying node.js at scale
summary
Starts as a MEAN monolith that dies under real load because Mongoose populate triggers hundreds of extra queries and unclosed file descriptors exhaust the OS until SSH fails; switching MongoDB to Postgres and replacing pipes with pump to close streams fixes some problems. Team growth forces a move to migrate monolith to microservices, using gossip/ SWIM, consistent-hash rings and a local tool called Fuge. Production stack ends up with ringpop + hyperbahn for app-level sharding and TChannel for RPC/tracing; Uber runs ringpop at ~20M TPS, but documentation and operational complexity remain challenging.
AI-written summary · about
chapters (33)
- 0:00Intro
- 1:49Choosing libraries
- 3:21Git Clone
- 3:36MongoDB
- 7:28Amazon
- 8:20File descriptors
- 10:11Back online
- 10:38The big problem
- 13:27The happy face
- 14:19The desperate thing
- 14:47Not the best solution
- 15:45Refactoring
- 16:19Microservices
- 16:35Microservices Architecture
- 17:54fuge
- 18:15Q interaction
- 18:59Service discovery
- 20:24Running on HDD
- 21:33Swing
- 25:49Update service topology
- 27:02shard
- 28:28secret
- 29:13ringpop
- 30:06sharding
- 31:18hyperbond
- 32:11circuit breaking
- 32:45kill HTTP
- 33:07TCP interpretation
- 33:34TCP documentation
- 34:18AWS Lambda
- 35:28Functional reactive programming
- 35:53Dont be too late
- 36:16Open Source
related talks
20mBeing Uber with Node
22mNode.js at UBER
27mScaling a Rails Monolith w/Event Driven Microservices
39mGrow with Less Pains - Meshing From Monolith to Microservices
31mKeynote Panel: Node.js at Massive Scale
24mArchitect for Scale - My Learnings in Node.js (A case study)
56mFrom Homogeneous Monolith to Heterogeneous Microservices Architecture
52mMicroservices the Lean Way NodeJS