Kubernetes as a Substrate for ATLAS Compute
summary
ATLAS integrates Kubernetes as a native compute substrate using Harvester plugins and Kubernetes Job controllers to run production and interactive workloads. Clusters scale on-demand (demo reaches 100k vCPU across ~2,000 nodes on GCP, processes ~100 million events) while tolerating 1% job failure on spot VMs. Software mounts use a CVMFS daemonset; EKS/GKE provide heterogeneous nodes (Graviton ARM, GPUs). Interactive work uses a Jupyter + Dask interactive facility that scales to thousands of workers and completes analyses in minutes.
AI-written summary · about
chapters (6)
- 0:00Start
- 5:26ATLAS computing flow
- 10:23ATLAS Distributed Computing
- 17:20Elastic cloud scale out
- 19:35Integration of heterogeneous architectures
- 30:57Conclusions
related talks
27mHarnessing Kubernetes for Scalable AI/ML Workloads
41mBuilding & Operating a Large-scale HPC AI Cluster on Kubernetes
31mManaged Kubernetes — Next Gen Academic Infrastructure?
35mKubernetes journey
08mLightning Talk: Orchestrating Kubernetes Clusters on HPC Infrastructure
07mKeynote: High Performance Computing on Google Kubernetes Engine
38mKubernetes for AI Workloads – What Works and What… Doesn’t
24mDynamic Big Data in the Cloud with Kubernetes