Keynote: gRPC, A Foundation for AI
summary
Google Cloud Storage uses gRPC to bypass frontend hops and unlock higher sustained bandwidth and more predictable tail latency, driving millions QPS and tens of terabytes of traffic in production. A GA gRPC API yields ~20% median improvement on 100 MB downloads (C++/Java). A new zonal SSD product, Rapid Bucket, exposes streaming RPCs (ByReadObject, ByWriteObject) that give submillisecond reads/writes and appendable objects, and updating GCSFS to the streaming API yields roughly 5x reads and 3x writes, cutting a real training workload on A4 GPUs by 23%.
AI-written summary · about
chapters (14)
- 0:00Introduction and speaker intro
- 1:06gRPC journey with GCS
- 2:13Bandwidth demands and REST limitations
- 3:40Direct connectivity benefits of gRPC
- 5:07gRPC adoption and performance gains
- 6:34Rapid bucket: gRPC-native storage
- 8:44AI workload speedups with Rapid Bucket
- 10:55Technical details of gRPC protocols
- 13:05Getting started with gRPC in GCS
- 14:10Q&A: GCS-specific API and connectors
- 16:16Q&A: Load balancing and streaming
- 18:11Q&A: Client-side load balancing
- 19:56Q&A: API stickiness and AI workflows
- 22:07Closing remarks and thanks
related talks
26mShould You Use gRPC in Your AI Tooling Stack?
18mgRPC and AI
35mPanel: AI's Real-Time Engine: gRPC, WebSockets, or SSE?
17mKeynote: Programming the Cloud as Easily as Your Laptop
15mKeynote: From Platforms to AI Factories. Has Kubernetes Solved It?
01h 44mDay 1 Keynote: Building a Cloud for Everyone
01h 20mParis Keynote
36mCeph Object Storage Roadmap Session at Cephalocon 2025