Using Temporal Workflows for Seamless Dev + Research Collaboration & Continuous Algo D
summary
Temporal is used as the backbone to run production ML pipelines where Temporal workflows hold orchestration and researcher-written activities are reusable, retry-safe functions. Integration time for a new algorithm drops from two–four weeks to under a day; iterative updates that used to take one–two weeks now deploy in under an hour. A live demo shows durable, replayable runs with traceable inputs/outputs and retries. The team runs workers in their VPC, keeps workflows deterministic, and calls this pattern zero-ouch integration.
AI-written summary · about
chapters (20)
- 0:00🚀 Build Stuff Conference – Get Your Tickets!
- 0:24Introduction: Why Engineering and Research Need Better Collaboration
- 1:40The Growing Complexity of AI Tooling and Workflows
- 3:20What Research Teams Actually Build: ML Pipelines in Practice
- 5:30The Core Problem: Translating Research Code Into Production
- 8:10Why Integration Becomes Slow, Painful, and Dependency-Heavy
- 10:20Different Collaboration Models: Researchers, Developers, and ML Engineers
- 13:10The Main Bottleneck: Research Playground Code vs Production Code
- 15:20First Attempt: Wrapping Algorithms as APIs and Why That’s Not Enough
- 17:10Enter Temporal: Workflow Orchestration for Durable Production Systems
- 19:00What Temporal Is and Why It Solves This Class of Problems
- 20:40Workflows, Activities, Workers, and Task Queues Explained
- 22:20How Roy’s Team Uses Temporal for Research Algorithms
- 24:00Demo Walkthrough: Running and Observing a Workflow in Real Time
- 26:00Splitting Responsibilities Between Researchers and Engineers
- 28:10The “Zero-Ouch Integration” Approach
- 30:00Results: Faster Deployments and Continuous Iteration
- 31:30Lessons Learned About Activities, Contracts, and Dependencies
- 33:00Future Improvements and Workflow Design Best Practices
- 34:20Final Takeaways: Making AI Collaboration Smoother and Faster
related talks
56mOpening Keynote
38mHow to Leverage Temporal for Agentic Orchestration and Workflows
33mFrom Prototype to Production: Making Agentic AI Systems Reliable
30mOrchestrate Your Business Workflows with Temporal
23mDesigning a Workflow Engine From First Principles
33mLessons learned using Temporal to run Temporal Cloud
26mTemporal: A Better Way to Build Modern Applications
41mOrchestrating Data Pipelines at Scale with Temporal