Do We Still Need People To Write Database Systems?
summary
Survey of two decades of analytic DBMS engineering recounts concrete advances — columnar storage, disaggregated storage, vectorized execution, then 2010s JIT compilation and UDF inlining — and evaluates machine‑learned components like learned indexes and learned cost models. Describes an autotuning system (OtterTune) that adjusts knobs via online models and reports ~3x–4x throughput improvements on RDS Postgres/MySQL. Lays out practical gaps: fail‑safe fallbacks, explainability, human override policies, transfer learning, and a roadmap toward a perception/action/planning self‑driving database.
AI-written summary · about
chapters (13)
- 0:00Introduction
- 4:36Columnar Storage
- 6:42Dis disaggregated Storage
- 9:01Vectorized Execution
- 11:12Query Compilation
- 13:45UDF Inlining
- 18:08Learn Components
- 25:39Automatic Configuration Tuning
- 28:25Challenges
- 31:50High Level Question
- 34:49Self Driving Database System
- 37:34Uncovering System Designs
- 41:55Questions
related talks
14mALERT: Machine Learning-Enhanced Risk Estimation for Databases Supporting...
18mBuilding a Scalable, Resilient System for Real-Time Fraud Detection
23mOpening remarks and Key Open Source Databases Strategies That Shape 2022
25mEnsuring Authorized Updates in Multi-user Database-Backed Applications
39mSelf-Service Analytics using LLMs
33mDriver for Data-Driven Change
51mEndogenous Threats to the Physical Integrity of DBMSs
33mAutonomous Database: taking Oracle's most secure database to the next level