AI Training Orchestration Evolution with Serverless Building Blocks
summary
Meta describes replacing a monolithic workflow engine with composable serverless pieces that cut ML training friction: a decoupled Action Service launches CPU, GPU and local jobs; a push-based event router replaces polling and reduces end-to-end training latency; shard-scoped MWFS decider+executor covers ~85% of patterns and supports multiple schedulers. Fblearner runs on top, over 1,100 teams use the platform in shadow before rollout to meet strict SLAs, and they are adding primitives like loop and map for richer pipelines.
AI-written summary · about
chapters (13)
- 0:00AI training workflow challenges
- 1:11AI training workflow complexity
- 2:38Limitations of RunFlow
- 4:25Vision for flexible workflows
- 5:49Meta Workflow Service (MWFS)
- 7:55MWFS workflow orchestration
- 9:42AI orchestration with MWFS
- 12:59AI model training observability
- 15:10Execution optimization with events
- 16:37Safe system extensibility
- 17:42Why build MWFS
- 19:09MWFS as workflow foundation
- 20:37Future workflow vision
related talks
38mServerless Architecture for Machine Learning
23mBuilding Agentic AI Systems with AWS Serverless
29mGoing Serverless
28mThe Brave New World of serverless computing
42mEnterprise Serverless Adoption. An Experience Report
15mServerless & Agentic AI: Better Together
51mChoreography vs Orchestration in serverless microservices
39mServerless at the end of the Universe