Intro: Dragonfly
summary
Dragonfly is an open-source P2P image and file distro system that Alibaba runs at cluster sizes up to 10,000 nodes, using a CDN+P2P mesh with DFDaemon and DFGet agents and a super node scheduler that assigns block-level downloads. Tests show a 750 MB layer stays ~18s per node under load versus registry-only times that explode; production use at Alibaba and ~50 adopters reduces bandwidth and stabilizes pulls. Integrates with Harbor and preheat APIs; cross-cloud replication and some enterprise features remain proprietary or under development.
AI-written summary · about
chapters (8)
- 0:00Introduction
- 2:56History
- 9:56Current Situation
- 10:46Performance Comparison
- 13:43Community
- 14:55Image Distribution
- 23:03Dragonfly Adoption
- 29:24Roadmap
related talks
30mDragonfly V2.5.0 - Intro, Updates, Data Distribution in AI Infrastructure
34mThree Approaches to Speed up Image Distribution in Cloud Native Era
30mFaster Container Image Distribution on a Variety of Tools with Lazy Pulling
22mAccelerate the Image Distribution for DADI Using Intel DSA
30mData governance and distribution with Dat
34mFast Image Pulls Using IPFS And Opportunistic Caching
27mDecentralized Storage with IPFS: How does it work under the hood?
13mARG / Estuary.tech