Large Synoptic Survey Telescope: From Cloud Native to Dark Energy
summary
Qserv is a petascale distributed DB built to hold 47 billion objects and about 9 trillion detections by sharding the sky into 20,000 shards across many nodes (roughly 5 TB per node in tests). Build pipeline uses Travis → Docker Hub CI with mini-kube integration tests; deployments run on bare metal OpenStack and GKE. The system loads 250 TB in lab tests, runs on Kubernetes with PVs and statefulsets, and shows portability and scaling though GKE can be ~25% slower than tuned bare metal.
AI-written summary · about
chapters (12)
- 0:00Start
- 2:09LSST in short
- 2:5580+ PB of astronomical catalog
- 4:50Data
- 9:58Qserv design
- 13:46Automated Qserv deployment
- 14:39Automated deployment: Cloud Native
- 18:18K8s + Microservice features
- 25:40The killer feature: workload portability
- 26:17Qserv deployment schemes
- 28:34Public cloud: pros and cons
- 31:12On-premise: pros and cons
related talks
24mScaling to Billions of Internet Observations per Day
47mCloud Native and Sustainable, Reproducible Scientific Computing
14mKeynote: Science at Light Speed: Cloud Native Infrastructure for Astronomy Workloads
24mSKA Observatory: Reaching to the Stars and Beyond
42mScaling Resilient Systems: A Journey into Slack's Database Service
46mBig databases, many servers, data center or cloud - it's all here!
30mStateful Superpowers: Explore High Performa...
31mScaling Autonomous-Driving R&D: A 100 PB, 100 Billion-File Open-Source Elastic File...