Container engines in High Performance Computing: some usage stories from CINECA
summary
CINECA describes four production uses of container engines in HPC: three deployments using Singularity (running Alphafold as a container-only distribution, packaging Flac3d to avoid Debian/CentOS mismatches, and rebuilding an iRODS client for PowerPC) and one using Podman to run GitLab CI runners on the Marconi 100 system. Containers are used to solve OS and licensing mismatches, required recompilation and repackaging for PowerPC, expose GPUs through a VM for CI, and run reproducibly at cluster scale up to 3,500 nodes.
AI-written summary · about
chapters (19)
- 0:00Container and HPC intro
- 1:12CINECA HPC mission
- 2:33HPC and Cloud at CINECA
- 4:38Leonardo cluster features
- 6:02Docker limits in HPC
- 7:47Singularity & Uptainer security
- 9:12Singularity vs Uptainer
- 10:59Singularity benefits for HPC
- 12:05AlphaFold containers case
- 14:31Flac3D containers case
- 16:00Ecommerce on Galileo 100
- 19:31Container benefits overview
- 20:12Second part intro
- 20:55Podman vs Docker
- 24:32What is CI
- 27:48GitLab CI pipeline
- 30:43Podman runner setup
- 32:32GPU in CI/CD containers
- 33:22Q&A
related talks
47mCloud Native and Sustainable, Reproducible Scientific Computing
39mThe future of infrastructure will be containerized
01h 02mContainers in Production: It’s Like Orchestrating Cats
23mImplementation Challenges: From HPC to Containers in the Academy
37mRunning Mixed Workloads on Kubernetes at the Institute for Health Metrics and Evaluation
08mContainerization – It’s Not Just for CPUs Anymore: Introducin...
39mHow we learned to love the Data Center Operating System
19mKeynote: Building Bridges: Cloud Native and High Performance Computing