Speed up Your ML Workloads With Kubernetes Powered In-memory Data...
summary
Demonstrates a distributed in-memory cache of Iceberg tables deployed as a leader/worker set on Kubernetes that shards files into Arrow-formatted shards using DataFusion, serves them with Apache Arrow Flight zero-copy streaming, and plugs into Kubeflow Trainer V2 via a dataset initializer. Claims: deserialization and IO drop dramatically, GPUs stop idling and throughput increases (demo fine-tunes an LLM with PyTorch, Hugging Face tokenizer, PyArrow data loader). Cache lifecycle ties to the train job today; roadmap adds reusable cache and catalog-based sizing.
AI-written summary · about
chapters (12)
- 0:00Introduction
- 1:04GPU inefficiency in ML pipelines
- 3:11Kubernetes and Kubeflow for training
- 5:00Challenges in sharding datasets
- 7:51In-memory caching architecture
- 10:38GPU workers with cached data
- 12:23Flight framework for streaming
- 13:27Cache integration with Kubeflow
- 15:34Live demo setup
- 18:41Running training with cache
- 20:51Benefits and future roadmap
- 23:19Q&A
related talks
26mUnlocking Scalable Distributed Training With Arrow Data Cache on Kubernetes
33mBridging Big Data and Machine Learning Ecosystems: A Cloud Native Approac...
34mLeveraging Apache Arrow for ML Workflows
31mBuilding a Data Infrastructure for AI/ML
25mOvercoming I/O Bottlenecks in LLM Training with Open-Source Distributed Caching
31mThe Future of Data on Kubernetes: From Database Management To AI Foundation
31mOpen Standards and Lakehouse AI Development
31mGotta Cache 'em All: Scaling AI Workloads With Model Caching in a Hybrid...