Analyzing Terabytes of Data with Google BigQuery
summary
Product externalizes Dremel and ColumnIO to run interactive analytics over billions of rows, storing data in column-oriented storage so queries read only needed columns and exploit compression. Demo runs a Wikipedia query that scans ~37 GB in ~5 seconds (another 400M-row example completes in ~30s), supports small joins by shipping right-hand tables <8 MB to all nodes, recommends denormalized tables, charges by data read, and only supports near‑real‑time appends (batched, not subsecond).
AI-written summary · about
chapters (21)
- 0:00Introduction
- 4:55Big Data at Google
- 6:08What is BigQuery
- 7:50Load Data
- 12:49Normalize Data
- 13:33Data Format
- 14:28Running Queries
- 17:53BigQuery Architecture
- 20:21Regular Expressions
- 21:01Joints
- 24:44Limit Clause
- 25:21Sampling Rules
- 26:01Top Command
- 26:45Why not MapReduce
- 28:06Summary
- 28:36BigQuery vs MapReduce
- 30:31Pricing Model
- 31:25Query Optimization
- 34:13columnstore
- 36:04near realtime data
- 40:08use cases
related talks
45mBuilding a Data Warehouse
52mBuilding the World's Largest Enterprise Data Warehouse with BigQuery
31mExploring open data with BigQuery
35mScaling Interactive and Insightful Dashboards with Data Studio and BigQuery
33mBigQuery DeepDive: dissecting Google's Data Warehouse
33mManaging IoT Storage with Google's Cloud Platform
47mFun with BIG Data with various databases
27mNotorious B.I.G. Data: Wins and fails when building a big data platform in the cloud