Skyplane: Optimizing Transfer Cost and Throughput Using Cloud-Aware Overlays
summary
Skyplane is a system for fast, low-cost bulk transfers between cloud object stores that probes cloud network throughput, models routing with mixed-integer linear programming, and provisions ephemeral gateway VMs to execute overlays. It finds trade-offs between egress fees (9–19¢/GB, intra-zone 1–2¢/GB) and performance, uses parallelism to bypass provider throttling, and reports up to 4.6× intra-cloud and 5.0× cross-cloud speedups versus vendor tools; open source and pip-installable.
AI-written summary · about
chapters (12)
- 0:00Intro
- 1:06The problem of "data gravity"
- 2:37What is Skyplane?
- 5:34Direct internet path between clouds are often slow
- 6:33overlay routing to circumvent slow links
- 7:24Insight #1: classic overlay routing is not designed in the cloud
- 9:21parallel VMs per region to avoid provider throttling
- 10:22parallel TCP connections to improve goodput
- 12:05Evaluation: End-to-end comparison against cloud providers
- 13:27Evaluation: Comparison to Resilient Overlay Networks
- 13:51Evaluation: Visualizing the cost-throughput space
- 14:51Open-source adoption
related talks
52mSky Computing- 52mJoint Keynote Address
16mSky Computing: Charting a New Course in AI Cloud Infrastructure
18mCloudcast: High-Throughput, Cost-Aware Overlay Multicast in the Cloud
13mSpace Clouds
14mSkyline: A Cloud Centric Internet Monitoring Engine
10mLightning Talk: Crossplane - An Extensible Approach in Distributing Workloads - Jason Bonafide, Dell
45mCloud Computing: Cloud in a cost-effective way