Container-enabled Asyncio is All You Need (to Build Pythonic AI Workflows at Scale)
summary
Claims that asyncio + Kubernetes is sufficient to coordinate and scale AI workflows, demonstrating concrete primitives—task groups and semaphores, bounded concurrency, timeouts, queues, streaming and retry logic—applied to LLM calls, retrieval fan‑out and rate‑limited APIs. Shows a working Flyte 2 reference implementation that runs locally on k3s, provisioning about ten pods to execute predict tasks in parallel, handling partial failures and OOM isolation, and even a containerized agent example that assembles “sandwiches” successfully.
AI-written summary · about
chapters (9)
- 0:00Async Python in AI workflows
- 1:51AI workflows at scale
- 3:56Async IO sufficient for AI
- 8:11Async IO fundamentals
- 15:13Async IO patterns for AI
- 21:36Containers for scaling
- 23:04Flyte 2 case study
- 28:01Trade-offs and mental model
- 29:26Call to action
related talks
34mOrchestrating Generative AI Workflows to Deliver Business Value
01h 24mRobust and End-to-End Cloud Native Machine Learning & Data Processing Platform
25mMLOps with Flyte
17mAI TechOps: Data and Infra for AI Foundation to Scale and Manage 1000s of Cohesive Agents
25mBuilding AI Workflows: From Local Experiments To Serving Users
27mAI-Powered ETL Pipeline Orchestration: Multi-Agent Systems in the Era of Generative AI
47mEmbedding AI: Don’t Burn Your Fingers in the Heat of the AI revolution
42mLLMs on Autopilot: Running AI Agents on Kubernetes With Open Source Tools