Universal Access to All Knowledge: Decentralization Experiments
summary
Internet Archive experiments with decentralised storage for its 100 PB collection using Filecoin, Storj and Arweave and reports concrete results: a 4 GB MP4 streams from ~3,000 nodes via erasure coding that selects 88 fastest peers, WARC web crawl archives are pinned to Filecoin/IPFS via web3.storage, and longterm cost modeling shows S3 at roughly $650 per TB for “forever” vs much lower Filecoin economics. Remaining gaps are indexing/metadata, encryption and ACLs, and scalable discovery for billions of captures.
AI-written summary · about
chapters (8)
- 0:00Decentralizing Internet Archive
- 1:48Spiritual reasons for decentralization
- 4:40Practical decentralization reasons
- 9:09Demo streaming media
- 13:03Demo web archiving with Filecoin
- 18:32Challenges in decentralized archiving
- 21:04Web memory forever?
- 21:49Q&A
related talks
17mDecentralized Technologies at the Internet Archive
17mPermanent Decentralized Storage Landscape in 2023
28mOpening Remarks / Filecoin: What's Now Possible
15mThe State of Filecoin
25mBuckets of Data: Strategies for Leveraging Off-chain Data Storage
37mStorage Provider Overview
24mOpen Source and the Decentralized Web
54mIPFS & Filecoin 101