Optimizing Cloud Compute Resources with Spare Cores
summary
Open source Spare Cores tracks over 2,000 server types and runs benchmarks that produced more than 800,000 scores, updates prices across 300+ availability zones with about 250,000 price checks every five minutes, and stores a 300 MB SQLite metadata dump. A ~1,000 line Python runner uses Pulumi to spin up machines, an inspector collects hardware and benchmark outputs in public GitHub repos, 30,000 GitHub Actions and 900 releases automate ETL, and users report a 65% cut on a $5,000 monthly AWS bill.
AI-written summary · about
chapters (10)
- 0:00Introduction and motivation
- 1:13Spare Core tooling and early use
- 3:19Data coverage and features
- 6:31Challenges in data collection
- 9:04Vendor pricing complexities
- 10:28Inspector and benchmarking
- 11:50Runner and infrastructure
- 13:58API and web interface
- 14:40Roadmap and acknowledgments
- 15:05Q&A
related talks
22mHarnessing Spare Cores to Breeze Through Cloud Compute
34mSELECT instance FROM cloud WHERE workload = ? ORDER BY cost
33mUncover Kubernetes cost-optimization trends
49mBuilding a zero-waste compute infrastructure on
39mWhat would you do with 10^6 cores? High Performance Computing in the AWS Cloud
36mIntroduction to High Performance Computing on Google Cloud Platform
42mBuilding a green supercomputer in the cloud
10mLightning Talk: Does Cloud Elasticity Pay off for HPC Workloads?