Scaling Down: Running Large Sites Locally
summary
A practical account of running a huge Django site locally by building an internal tool called Bay that layers on Docker, supports dev modes for live-mounting code, and runs dependency graphs in parallel so developers avoid booting 150 containers at once. It trades full image rebuilds (previously hours) for cached, centrally built images and live mounts, cuts worst-case monolith rebuilds toward ten minutes, adds a Python init to track health and exit-on-failure, and publishes the project on GitHub.
AI-written summary · about
chapters (19)
- 0:00Intro
- 1:26One big Vagrant VM
- 2:05150 Docker containers
- 3:45docker-compose
- 4:15bay
- 4:35Runs against Docker API
- 5:55Running subsets of the site Developer code "mounting"
- 7:01Containers & Links
- 10:54Did it boot yet?
- 12:58Runs parallel processes (and exits on any) Cleans up zombies
- 13:51TCP waits (socket opens) HTTP Waits (200 status check)
- 15:13Rebuilding is Bad
- 15:44Rebuilding core is about 10 mins It would be bad to do this on every tiny change.
- 18:57Our dependencies change every few hours You can see how the build caching is not so useful.
- 20:13Docker images are portable Just need to build them in the first place.
- 22:06Automated image building Plus versioning/tagging
- 23:22Running images remotely We really need to soon
- 24:32Running whole stacks remotely Potentially for integration testing/acceptance
- 25:33Trying not to reinvent the wheel I don't have the time to maintain another kubernetes/docker swarm/nomad!
related talks
13mBuilding reproducible dev environments with Docker
28mDocker-Composing Your Way to a Better Development Environment
38mContainerless Django: Deploying without Docker
14mSmall Containers? or Small Images?
34mAn Intro to Docker for Djangonauts
24mSpeeding up Your Docker Image Build
37mLessons learned from dockerizing complex Web Apps
29mCan packaging improve Django deployments?