Lessons learned from refactoring multi terabyte databases for the last 10 years
summary
Ten years of refactors on SQL Server and other engines hit limits and teach concrete patterns: use IDEs and tests (for example DataGrip finds dynamic‑SQL renames), deploy with ordered versioned scripts, and prefer small step changes. Moving 10% of columns yields ~11% more useful cache, 35% → ~50% memory gain; one case dropped one index saved 40% of the database (several TB). For large moves use partitioned tables, bulk loads, parallel‑run and event replay (rebuild by partition after moving data); a three‑day offline migration finished two hours before reopening.
AI-written summary · about
chapters (19)
- 0:00Intro
- 2:26Development Tools
- 3:39SQL Server Features Which Helps Refactoring
- 6:07User Access Documentation
- 6:57Document Interfaces Through Tests
- 8:51Step-By-Step Examples
- 11:37Deploy Scripts Versioning
- 12:41Benefits of Deploy Scripts
- 14:08Sandboxes
- 17:09An Event Log Before Database
- 23:23Internal Fragmentation
- 27:44Data Migration/Synchronization Techniques
- 33:21Parallel Run
- 36:13External API Pros
- 39:30Constraints
- 40:32Split Database First
- 40:46Split Code First
- 47:04Replace One-To-Many With Associative Table
- 48:59Data Exchange
related talks
54mLessons learned from refactoring multi terabyte databases for the last 10 years- 54mLessons learned from refactoring multi terabyte databases for the last 10 years
38mEfficient Reliable Database Migration: A Legacy Innovation Story
49mHidden Gems in SQL Server 2022 Database Engine
52mWriting T-SQL code for the engine, not for you "more demos" by popular demand
49mAssessing and Optimize Enterprise SQL Server Environments
51mDealing with Very Large Tables in SQL Server
28mSmall Data: Databases in the real world