Building Production Applications Using Go
summary
Argues that using sqlite with Go can replace many client‑server setups: recommends go-sqlite3 (mattn) as the default driver, enables WAL journaling and sets busy_timeout=5000 and PRAGMA foreign_keys=ON, and uses in‑memory databases for testing. Reports single‑node point queries around 18µs vs ~900µs for same‑region Postgres, 10,000 tiny tests finish ~1.5s with a tailscale build while mattn’s global lock slows parallel tests to ~3s. Offers durability paths: simple hourly backups, lightstream to S3 for continuous WAL shipping, or rqlite/dqlite clusters.
AI-written summary · about
chapters (21)
- 0:00Intro
- 0:17About me Ben Johnson
- 1:26Client/server & cloud databases
- 3:49What applications are good for SQLite? • Originally used by embedded software • Improvements in concurrency have made it more general purpose . Great for read-heavy workloads • Small to moderate request loads (100's of requests/sec) . Honestly, most applications are good candidates
- 6:35Connecting to SQLite import
- 7:27Configuring SQLite: Journal Mode • Journal mode configures how SQLite writes transactions . You almost always want it as "WAL" (write ahead log)
- 8:26Configuring SQLite: Busy timeout • Busy timeout sets how long write transactions will wait to start . If unset, writes will fail immediately if another write is running . I find that 5 seconds is a good amount of time.
- 9:09Configuring SQLite: Foreign keys • For historical reasons, foreign keys are not enforced by default. . Please use foreign keys.
- 10:00SQLite Type System
- 13:27Benchmarking SQLite tests
- 14:35Parallelizing in-memory SQLite
- 17:12The data durability spectrum How safe is your data?
- 19:07Improving data durability Trade-off of cost, complexity, & performance
- 20:14No data loss is good ...but not all data loss is catastrophic . GitLab lost 6 hours of data in 2017 .Postgres primary replica configuration • Data deleted by operator error
- 22:39Option #1: Regular backups Data durability in SQLite
- 23:59Litestream
- 26:28Clustered SQLite
- 28:17Performance between SQL databases How to choose?
- 29:12Benchmarking network latency point query performance 1000
- 31:48Horizontally scaling SQLite Thinking the unthinkable
- 37:02Conclusion Improved production experience
related talks
30mUse SQLite in production
47mA Journey to Postgres Productivity with Go
01h 07mHow (and why) to run SQLite in production
01h 06mrqlite: The Distributed Database Built on Raft and SQLite
32mA Journey to Postgres Productivity with Go
45mSQLite on Rails: Everything you need to know
49mWhen it all GOes right
23mFANTASTIC DATABASES AND WHERE TO FIND THEM