Writing High Performance Databases in Go
summary
SkyDB and Bolt are written in Go with selective C integration via cgo and LuaJIT/LLVM for hot paths; design uses mmap, a B+ tree, MVCC and zero-copy reads. Batched commits show large gains (1→10 ≈ 9×, 100 ≈ 45×, larger batches ≈ 90×) and transaction coalescing trades latency for huge throughput. An unsafe direct-memory layout bypasses encoding and reaches 20 million values a second on reads, at the cost of portability and safety.
AI-written summary · about
chapters (13)
- 0:00Writing databases in Go
- 1:30Two database types compared
- 2:53Why Go over C or Java
- 5:40Low-level Go optimizations
- 6:00Bolt design with mmap
- 7:21Batching writes for speed
- 8:46JSON encoding cost
- 10:14Zero-copy data mapping
- 13:22Performance optimization layers
- 15:15Mmap and disk trade-offs
- 17:03Go LMDB wrappers compared
- 18:05Designing intuitive APIs without sacrificing performance
- 19:30Q&A
related talks
39mBuilding Production Applications Using Go
35mFast on paper, slow in reality: what we got wrong about performance
37mArchitecture Guidance
01h 27mFlash Talks
37mBuilding a Cloud-Native SQL Database
17mComparing DBs with Golang in the cloud
15mTime Series and FoundationDB: Millions of Writes/s and 10x Compression in 2000 Lines of Go
52mCockroachDB