Data Warehouses and Multi-Dimensional Data Analysis
summary
Builds a star/snowflake data warehouse from Rails transactional tables, loads it via ETL (mentions Kiba and concurrent-ruby) and queries it with a Mondrian OLAP engine exposed through a JRuby gem. Tests use a 6 million row fact table and show ad‑hoc SQL aggregations taking ~18–25s while Mondrian caching drops repeated queries into 10 milliseconds; columnar analytical stores (Vertica/Redshift) cut cold queries to ~9s and warm to ~1.5s. Recommends JRuby for parallel ETL, bulk loads for column stores, and considering analytic DBs once facts exceed ~10 million rows.
AI-written summary · about
chapters (29)
- 0:00Intro
- 0:22Latvia knocks US men out of Olympic beach volleyball
- 2:24Database schema
- 4:31Generated SQL
- 6:07Performance slows down on larger data volumes
- 7:21Dimensional Modeling
- 8:42Data Warehouse Star schema with fact and dimension tables
- 10:31Snowflake schema
- 10:59Data Warehouse Models
- 11:28Load Dimension
- 12:20Load Facts
- 13:38Two-Dimensional Table
- 13:55Multi-Dimensional Data Model
- 14:47Dimension Hierarchies
- 15:18Time Dimension
- 19:31MDX Query Language
- 20:13Results Caching
- 21:29Additional Attribute Dimension
- 22:23Dynamic Attribute Dimension
- 23:32Calculation Formulas
- 25:09ETL process
- 26:20Ruby Tools for ETL
- 26:36Kiba example
- 27:10Multithreaded ETL
- 30:06Benchmark!
- 30:59Traditional vs Analytical Relational Databases
- 32:47Row-based Storage
- 33:36Columnar Storage
- 35:29Analytical Query Performance
related talks
30mBusiness Intelligence with Ruby
28mLightweight Business Intelligence with Ruby, Rails, and MongoDB
29mBig Data Enterprisy Analytics App and Ruby
18mDrowning in Data but Searching for Insight? Delivering Sophisticated Analytics with Open Source
58mSuper Jumbo Delux
43mBreaking Down Data Warehouses: What to Know Before You Choose
36mData Management with Ruby
27mBuilding a PostgreSQL data warehouse