Ownership: A Distributed Futures System for Fine-Grained Tasks
summary
Ownership claims a decentralized runtime that shards metadata by object owner, trades global persistence for simple recovery, and achieves 1ms latency per task on fine-grained millisecond tasks. Implementation in Ray shows horizontal scaling and a live video stabilization demo where per-frame tasks pipeline, failures trigger owner-driven reexecution and lightweight application-level checkpoints, and owner failure is handled by fake-sharing children to restart work while bounding duplicate execution.
AI-written summary · about
chapters (15)
- 0:00Intro
- 0:29RPC model
- 3:04Distributed futures today
- 3:38A distributed futures system for fine-grained tasks
- 4:24Distributed futures introduce shared state
- 5:04System requirements
- 6:16Our approach: Ownership
- 7:51Ownership: Challenges
- 8:33Distributed memory management
- 8:48Worker failure
- 9:02Owner failure
- 9:18Owner recovery
- 9:44Outline
- 10:33Evaluation: Online video processing with failures
- 11:22Conclusion
related talks
24mRay Internals: Object Management with the Ownership Model
28mA Deep Dive into Ray’s Scheduling Policy
16mDRust: Language-Guided Distributed Shared Memory with Fine Granularity, Full...
14mBuilding High Availability and Scalability Online Computing Applications on Ray
39mTrust Deterministic Execution to Scale & Simplify Your Systems
23mDesigning a Workflow Engine From First Principles
41mResource Fairness: Implementing (and Deprecating) a Lease Broker
24mShray: an Owner-Compute Distributed Shared-Memory System