Revolutionizing the Big Data Age With Compute over Data
summary
Claims a usable project called Bacalhau that runs Docker/WASM jobs where the data lives, cutting the need to move 100 terabytes across networks. Demonstrates IPFS for inputs, DuckDB/ pandas examples, and actually runs AlphaFold and YOLO on remote GPUs, with content‑addressed IDs, automatic placement and a verification step. Network is beta, aims to reduce ingress/egress cost and provide reproducible, verifiable outputs rather than just proposing a decentralized idea.
AI-written summary · about
chapters (10)
- 0:00Compute over data intro
- 3:39Big data cost challenges
- 6:10Decentralized data nature
- 9:25Requirements for compute systems
- 13:19How networks operate
- 18:16Auto-healing benefits
- 20:23Compute working group
- 21:48Choosing compute model
- 22:51Baka Yao network demo
- 26:50Future of data processing
related talks
24mData infrastructure to build bigger with less
17mEnabling Compute and Data Sovereignty with Infrastructure-Level Data Spaces
28mHerding Data to the Cloud
20mData in the Apache "big data" ecosystem
51mMotherduck: Data Analytics in the Post Big-Data Era
55mThe Great Data Debate 2022
16mLess but Better: Toward Simpler and More Efficient Data Engineering…
21mOpportunities in The Web3 Data Economy