Breakout Session
summary
MixRank moves from hundreds of GB to tens of terabytes by combining big machines and a cluster, tuning hardware and filesystems, using 70k IOPS and 3.7GB/s per server, half‑TB RAM and dedicated hosts, plus lz4 compression for ~40–45% space savings. Operations use partitioning by schema and time, materialized views, SQLAlchemy in Python and CTEs, and a pragmatic multi‑plan race that runs several plans concurrently and kills losers — a practical system that works in production.
AI-written summary · about
chapters (26)
- 0:00Introduction
- 4:14Scaling with Postgres
- 9:10Hybrid approach
- 14:33Scaling databases
- 14:58Postgres Query Planner
- 24:16Benefits of Pointer
- 25:09Resource Bloat
- 26:10Net Effect
- 26:49Intelligent Prediction
- 27:30Dynamic Operations
- 28:11Dynamic Compose
- 29:32Questions
- 30:28Cache
- 31:29Constant Factory
- 34:07Statistics Target Weight
- 34:39The Lesson
- 35:18Open Source
- 37:34Application Code
- 40:29Execution Engines
- 42:17Changing a Plan
- 43:56Better Own Extension
- 44:52Better Plans
- 47:37Gekko
- 50:47Query Hints
- 53:16Batch Updates
- 55:34Table Bloat
related talks
45mDealing with TBytes of Data in Realtime
22mIt’s all about the db - a journey to accommodate 100k events per second
48mAnalytics Architectures on Microsoft Fabric
30mCatching Waves with Time-Series Data
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
25mHow Sharekhan Scaled Trading Platform with Yugabyte's Distributed SQL
46mBig databases, many servers, data center or cloud - it's all here!
28mData @Scale Seattle- Brian Beach