Why Models Break Your Pipelines
summary
Claim: models break pipelines because they’re expensive, stateful, and hard to configure. Concrete evidence: Daft adds stateful UDFs (async, generators, batching) so one worker loads gigabytes of weights once; Gemma 3 runs at ~4B/9 GB while Kimmy K2 is ~1T params and needs 8×H200 / 632 GB just to initialize. Built result: a Daft VLM prefix‑caching provider avoids a global sort and shows a 50% speedup on batch inference, plus native Rust IO optimizations for S3.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 2:52Models break pipelines
- 7:52Understanding UDFs
- 14:39Stateful inference
- 22:10Large multimodal models
- 25:22Stateful UDFs in Daft
- 29:19Models in pipelines
- 33:16Q&A
related talks
26mOne Pipeline to Rule Them All: Unifying Multimodal and AI Data Processing with Daft
38mData Secrets from a Platform Engineer
21mMulti Runtime Serving Pipelines
18mUnleash your Private Datasets to Power up AI
26mMLOps for the rest of us- A poor man's guide to putting models in production
30mWhen Good Models Go Bad: The damage caused by wayward models and how to prevent it
17mAttack of the Bubbles: Straggler-Resilient Pipeline Parallelism for Large Model Training
37mUniversal AI: Execute Your Models Where Your Data (And Users) Are