The Great Data Debate 2022
summary
Cloud-first architectures push compute into databases so teams treat storage (Parquet, object stores) and Apache Iceberg as the canonical layer, claim vendors, enabling near-unlimited scale but forcing strict cost governance. Customers report big migrations reveal many unused workloads; one vendor says average customer spend rises sharply year over year. Practical fixes: query-acceleration to avoid copies, Git-style versioning for data (Nessie), automatic lineage and PI tagging for enforcement, and open table formats to let engines interoperate.
AI-written summary · about
chapters (11)
- 0:00Introduction and panelists
- 3:19Economic impact on data strategies
- 7:55Trends in database analytics
- 11:49CFOs and data cost control
- 19:31Software development meets data engineering
- 26:54Data mesh and data products
- 29:01Best of breed vs data fabric
- 33:18Metadata standards and interoperability
- 40:21Data modeling in modern analytics
- 44:14Data catalogs and governance
- 51:14Future investments and advice
related talks
33mThe advent of the open data lake
43mArchitecting for Speed & Scale
28mThe architecture behind modern cloud data warehouses
41mOn Data: Sky above, sand below, peace within
44mWhat's the Best Big Data Architecture for You?
21mUpgrading from the modern data stack to the modern data lake
47mCloud Data Management
02mThe Case for Open Data Architecture