DLSpec: A Deep Learning Task Exchange Specification
summary
Claims a reproducible DL task format that is model-, dataset-, software-, and hardware-agnostic, defines tasks as four manifest files (hardware, software, dataset, model) plus a reference log, and embeds pre/postprocessing as executable Python to avoid serializing intermediates. Authors implement a distributed runtime that parses manifests, downloads and checks remote resources, and runs streaming training/inference; they report specifying and running hundreds of DL tasks to validate repeatability and cross-framework, cross-hardware execution.
AI-written summary · about
chapters (12)
- 0:00DLSpec: A Deep Learning Task Exchange Specification
- 0:25Background
- 0:54Current Practice of Publishing DL Artifacts • Ad-hoc scripts and textual documentation to describe the
- 3:06DLSpec is based on a Few Key Principles
- 5:57DLSpec Design
- 6:29Hardware Manifest
- 7:22Software Manifest
- 8:02Dataset Manifest
- 8:42Model Manifest • Defines the logic to run a DL task and the required artifact
- 10:42Reference Log
- 11:21A DLSpec Runtime Consumes the Manifests
- 13:22A Runtime for Benchmarking DL Inference- MLModelScope
related talks
29mFormal Specification and Testing for Reinforcement Learning
01h 16mReproducible machine learning and science with python
38mSPONSOR WORKSHOP / Zhiyi Ma, Shagun Sodhani / Facebook
17mScaling DSpark Training Using vLLM, Speculators and Mooncake
28mIndustrial Strength DALLE-E:Scaling Complex Large Text & Image Models
32mReproducibility and versioning of ML systems
25mFine-Tuning Large Language Models With Declarative ML Orchestration
34mOpen Standards For Machine Learning Model Deployment