Indexing Ethereum Mainnet for Near Zero Cost
summary
Proposes adding get appearances in a block to RPC so nodes return every address seen per block, and demonstrates TrueBlocks’ toolchain: Shifra scrape to extract appearances (finding ~133% more address hits than a typical Web2 API), chunking sorted results, storing a manifest on IPFS, and publishing the manifest hash to a smart contract. Uses Bloom filters to shrink a 100 GB index to ~3 GB per user, claims near-zero ongoing cost (roughly $0.50/day or ~$500/year) and reproducible, permissionless indexing.
AI-written summary · about
chapters (11)
- 0:00Ethereum indexing challenges
- 1:39Missing RPC for account queries
- 4:09Address appearances in raw data
- 7:50Storing index immutably in chunks
- 10:17Publishing index via IPFS and contract
- 12:47Near-zero cost maintenance
- 13:51Motivation for public sharing
- 16:22Reducing storage with Bloom filters
- 18:06Benefits of get appearances proposal
- 20:57Web3-native data alternatives
- 21:18Q&A on IPFS availability
related talks
32mIndex, Query, Ship - HyperIndex + HyperSync Live
31mBlockchain + Tracking = Awesome Analytics
26mEthereum Blockchain Analytics
40mDeveloping Distributed Apps on the Ethereum Blockchain
01h 04mBlockchain, Smart Contracts, and Ethereum
16mTargeting Zero MEV in the Base Layer
26mLiving on Ethereum
31mEthereum from even more perspectives