Presto in the Cloud
summary
Presto runs as part of Qubole on AWS with ephemeral clusters that spin up and tear down, adding nodes in ~2 minutes via a modified StarCluster so queries scale. Query engine keeps intermediates in memory, uses codegen and vectorized execution, and avoids MapReduce. Hot S3 data is cached in Guava and on-instance SSDs; writes use parallel inserts and multipart upload. Using ORC files delivers the biggest gains — benchmarks show up to 22x speedups and sub-10s TPCH queries on a $13/hr cluster.
AI-written summary · about
chapters (25)
- 0:00Introduction
- 0:39Presto DB
- 2:33Cloud Compute
- 3:58Cloud Data
- 6:00Analytics
- 7:29Deep Analytics
- 10:17Whats Special
- 10:53High Performance
- 13:03Reading
- 14:02Vectorize
- 16:54Assumptions
- 17:51Qball
- 20:16Flexibility
- 21:28Shutting down clusters
- 22:04Dynamic resizing clusters
- 23:29Automatic configuration
- 25:42Query stages
- 27:30Nation table
- 31:25Hive MapReduce
- 34:14Pipelining
- 37:13Downscaling
- 39:14Cache Data
- 40:06File Format
- 43:00Performance
- 44:51Conclusion
related talks
50mPresto: SQL-on-Anything using Kubernetes
22mPresto for Your Fast, Open SQL Lakehouse
51mPresto Open Source SQL Engine for Massive Scale Data Lakehouses
22mIntroduction to Presto: The SQL Engine for Data Platform Teams
35mIntroducing Prestissimo, Presto's Next-Gen Query Execution Engine
48mContinuous Optimization for distributed BigData analysis
28mOptimizing Latency-Sensitive Queries for Presto at Facebook
36mQuantum & High-Performance Computing in The Cloud