🛠️ Filecoin data onboarding with Delta
summary
Delta is an API-driven Delta deal engine microservice (open source on GitHub) that automates CAR and ComP generation ~2s versus minutes, selects SPs, submits online or offline deals, transfers data, and tracks/repairs deals; demo shows cloning the repo, obtaining an auth token, and a curl POST uploading a 1 GB test file that returns a content CID and status. For many small files, Estuary/Edgar aggregates and batches them; for terabyte datasets, Delta DM manages replication, wallets, providers and includes a UI.
AI-written summary · about
chapters (10)
- 0:00Intro
- 1:33Getting data onto Filecoin (Part 4)
- 5:31Filecoin Data Tools
- 10:24What happens after uploading a file
- 11:36Optimization 1: CommP Generation
- 12:59Optimization 2: Infra scaling with FDI
- 16:02Deals with API
- 21:39Deals with Edge UR
- 24:30Delta Dataset Manager
- 25:21Dataset upload process
related talks
14mFilecoin - Building with the Filecoin Onchain Cloud
32mDesigning Data Infrastructure in the Age of Generative AI
38mGetting Started with Delta Lake
28mOpening Remarks / Filecoin: What's Now Possible
18mTokenizing the Data Economy
30mDelta Lake and Delta Sharing
16mThe Evolution of Delta Lake
01h 28mBuilding Reliable, Open Lakehouses with Delta Lake