Write-Through Object Cache for Fleets of Global Edge Clusters
Talkat Data on Kubernetes Day Europe 2025 · on Tuesday, 2025-04-01· at 11:15· Level 1 | Hall Entrance N10 | Room E·
summary
A write-through object cache runs on regional Kubernetes edge clusters using AIStore with OVN, MetalLB and egress IP, deploying AIS proxies and storage targets as daemonsets backed by local NVMe PVCs and multihomed networking. The cache is opportunistic and falls back to origin; it keeps consistency by writing to origin on session end. A three-region canary of 400,000 samples over two weeks shows 51% download throughput and a 35% reduction in download time, ~45% faster client uploads, and an expected reduction of hundreds of petabytes of egress annually.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 1:04Use cases
- 3:12Challenges for edge clusters
- 6:00Open source components
- 9:55Architecture explained
- 12:46Write-through cache choice
- 13:52Canary impact
- 17:05Alternatives considered
- 19:16Summary and Q&A
related talks
36mOptimizing Ceph Object Storage for Analytics and AL/ML Workloads - Overview and Future Plans
07mData Gravity and Kubernetes: Managing Large-Scale Data Ingest With Minimal Latency
29mOn Data Locality in Kubernetes
25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed...
14mBuilding a high-performance analytics ecosystem on Amazon EKS
32mStreamlining Data Access for with
57mDriving innovation and insight with storage
14mFalconFS: Distributed File System for Large-Scale Deep Learning Pipeline