10x smaller docker containers for Data Science
summary
Starts from a 1.7 GB baseline and actually reduces a PyData image with Matplotlib Scipy Numpy and Scikit-learn down to ~188 MB using concrete steps: use python3.8-slim, add a .dockerignore, inspect layers with dive and docker history, collapse RUNs to minimize layers, use multi-stage builds, build wheels with pip wheel and repair with auditwheel, set CFLAGS to strip debug info. Result is ~9–10× smaller but much slower to build (59 minutes vs ~5) and more complex to maintain.
AI-written summary · about
chapters (20)
- 0:00Introduction
- 1:07Minimal Docker images
- 3:16Docker ignore files
- 5:23Image size with Docker history
- 6:03Python slim base images
- 7:32Dive tool for layers
- 10:22Unnecessary files removal
- 12:28Minimizing layers
- 13:54Squashing layers
- 14:15Alpine Linux base
- 16:25Python wheels compatibility
- 19:33Compiling Alpine packages
- 21:40Removing build dependencies
- 22:23Multi-stage builds
- 23:47Fixing wheels with auditwheel
- 25:37Compiler flags optimization
- 27:02Duplicate libraries removal
- 28:24Size vs build efficiency
- 29:06Key takeaways
- 30:09Q&A
related talks
40mDocker and Python: making them play nicely and securely for Data Science and ML
42mMaking Docker & Python Play Nicely and Securely for Machine Learning & Data Science
01h 02mDocker for Data Science: Reproducibility and Deployment
26mUnravelling the Magical Act of DockerSlim Minifying Container Images
27mDocker Image Journey - How to Shrink a Docker Image
24mSpeeding up Your Docker Image Build
14mSmall Containers? or Small Images?
02h 28mWorkshop: Learn How to Dockerize Your Python Application