Scaling Webpack to Thousands of Concurrent Builds
summary
Warehouse.ai is an open source distributed build system that makes webpack reproducible by storing npm tarballs so npm install becomes deterministic the system uses NSQ as a pull queue to avoid DNS cache starvation and accepts idempotent build messages workers run in kubernetes pods with autoscaling based on node CPU to absorb Monday morning spikes the talk details npm Registry HTTP API quirks how to handle arbitrary user webpack configs where to persist minified versus source artifacts and why Lambda often hits disk and time limits for big installs
AI-written summary · about
chapters (17)
- 0:00Intro
- 0:11SCALING WEBPACK
- 0:43SENIOR DIR, UX PLATFORM @ GoDaddy
- 4:09FILES TO DISK
- 5:04CPU BUILD TIME
- 5:15MINIFICATION
- 5:28CONFIG lots of CONFIG
- 5:45DEVELOPERS WANT THE HOMER CAR
- 6:32WEBPACK service
- 7:13npm install
- 9:37consider a naive solution
- 11:01CONSIDER scale
- 11:50what about queues?
- 12:30characteristics
- 13:21ORDER DOES NOT MATTER
- 16:50WHAT IS SCALE?
- 17:42spikes within
related talks
16mRealtime Webpack - Pushing on-demand bundling to the limits
31mWe Started Using Webpack and It Took A While
40mBuilding a Platform: Webpack and the Future
48mBuild tools 101
01h 02mWebpack 5 Module Federation
51mMoving at the Speed of Thought
36mMicrofrontends: Scalability Savior or Unmanageable Maze?
32mWhat's New in Webpack 4