Keynote: Building Bridges: Cloud Native and High Performance Computing
summary
CERN describes replacing an online farm with Kubernetes since 2017, moving petabytes/sec of detector data down to tens of gigabytes and running a couple thousand node cluster that now sustains 300 pods per second. The Grid spans ~200 sites and treats HTC as loosely coupled jobs; the cloud-native solution adds queues, cluster-queues, cohorts and fair share. A scheduler prototype called queue (alongside Volcano and Unicorn) borrows public cloud capacity — a demo links a 10,000 CPU cores kubecon queue and briefly runs ~3.5k cores to drain pending work. API Priority and Fairness addresses earlier scheduling limits.
AI-written summary · about
chapters (6)
- 0:00Modernizing systems at CERN
- 2:29HPC and cloud requirements
- 6:04High throughput computing
- 7:55Batch scheduling evolution
- 12:28Scheduling demo and analysis
- 18:17Conclusion and initiatives
related talks
36mAn almost perfect match: cloud native and scientific computing
47mCloud Native and Sustainable, Reproducible Scientific Computing
27mFrom Supercomputing to Serving: A Case Study Delivering Cloud Native Foundation Models
14mKeynote: Science at Light Speed: Cloud Native Infrastructure for Astronomy Workloads
14mKeynote: The Big Leap: Conquering the Legacy System and Building a Scalable Cloud...
10mKeynote: Enabling AI with Cloud Native
08mLightning Talk: Orchestrating Kubernetes Clusters on HPC Infrastructure- 10mKeynote: Enabling AI with Cloud Native