Biggish Data With Rails and Postgresql
summary
Demonstrates that conventional ActiveRecord patterns break when tables hit hundreds of millions of rows and databases approach a terabyte of data, and gives concrete fixes: move the database to a beefy server — use a real damn computer — tune Postgres with tools like pgtune, front it with pgbouncer, switch expensive pagination to indexed range queries, use partitioning to drop old data, increase read‑ahead and RAM, and adopt streaming WAL backups so backups and deletes actually finish.
AI-written summary · about
chapters (39)
- 0:00Intro
- 0:48Sign Painting
- 1:16Cold Calling
- 1:34A Blessing in Disguise
- 1:48Becoming a Programmer
- 2:33Freelancer Skills
- 2:46Scale
- 3:07Biggish Data
- 3:48Data Size
- 4:00The Good News
- 4:37Live Query
- 4:59Page Data
- 5:42Summer of Disco
- 5:59Alpha Nerd 1978
- 6:19RealTime Computing
- 7:32Moores Law
- 8:00Exception Monitoring
- 9:02Database Performance
- 9:20Use a Real Computer
- 10:01Understand Queries
- 10:38Rows
- 11:00More Problems
- 11:30pagination
- 11:58range query
- 12:15index
- 12:43quicksort
- 13:05two big wins
- 13:21you guys deserve a cookie
- 13:33nothing could ever go wrong
- 13:54common problems
- 15:18readahead cache
- 16:32slow vacuuming
- 17:58database connections
- 19:52locking problems
- 21:27readonly replicant
- 22:14Auto partitioning
- 24:04Backups
- 24:59You can do this
- 25:32Slide deck
related talks
31mBiggish Data With Rails and Postgresql
30mPuny to Powerful PostgreSQL Rails Apps
01h 01mNew Approach to Resolve objectrelational impedance mismatch
48mHow to Tame A Mastodon: Lessons for PostgreSQL at Scale
36mPostgres at any Scale
39mPostgres & Rails 6 Multi-DB: Pitfalls, Patterns, Performance
30mManaging Postgres at scale: Challenges, Tools & Techniques
43mJust Use Postgres.