Cloud Native and Sustainable, Reproducible Scientific Computing
summary
Reproduces the Higgs analysis on a 70 TB dataset, scales from days to 90 seconds by bursting to public cloud with 25,000 cores managed via Kubernetes and container images, and shows container startup cut by using SGZ lazy pulling instead of full 10–100+ GB image downloads. Demonstrations include Crossplane provisioning VMs from Kubernetes, Q for priority queuing and preemption with graceful checkpointing, GitOps-driven GPU allocation, and a multi-cluster management plane running workloads across on‑prem and cloud sites.
AI-written summary · about
chapters (15)
- 0:00Cloud native scientific computing
- 1:23CERN's computing evolution
- 6:21Mainframes to distributed systems
- 8:51Cloud computing rise and impact
- 12:48Cloud Native Computing Foundation and Kubernetes
- 16:20CNCF project maturity
- 18:52Reproducing Higgs discovery
- 24:29Optimizing container images
- 28:42Demo: lazy-pulling images
- 30:49VMs with Kubernetes and Crossplane
- 34:46Batch processing with Kubernetes and Q
- 37:56Demo: Kubernetes batch queueing
- 38:39Multi-cloud workload management
- 43:03Future of cloud native computing
- 44:10Q&A: Kubernetes preemption
related talks
27mFrom Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models
19mKeynote: Building Bridges: Cloud Native and High Performance Computing
20mKeynote: Reperforming a Nobel Prize Discovery on Kubernetes
18mKeynote: Mind the Gap: Bridging Cloud Native Innovation with Real-World Use
16mA novel Cloud High Performance Computing infrastructure for user
43mCloud Native - effective computing at scale
20mHow the cloud changed the way I do analytics and research
21mKeynote: CERN Experiences - Ricardo Rocha & Clenimar Filemon