Columnar Storage + Python: Powering Modern Data Science and Analytics
summary
Columnar storage here uses MariaDB ColumnStore with extent mapping and distributed column partitions to cut IO dramatically (example: a 1B-row aggregate reads ~200 GB row-wise vs ~4 GB column-wise). A cloud instance on SkySQL holds 40 million rows of flight data; queries against the columnar table run without indexes (counts and averages return in ~200–300 ms). Python connector + JupyterLab with pandas and Plotly fetches and visualizes results live, and the demo successfully maps flight volumes by airport.
AI-written summary · about
chapters (9)
- 0:00Intro
- 3:02Overview
- 9:32Pluggable Storage Engines
- 14:15Storage Architecture
- 21:27Distributed Storage
- 25:58Walkthrough
- 28:28Connecting to MarieDB
- 32:48Python Notebook
- 38:38Outro
related talks
17mUsing MariaDB ColumnStore with Power BI for Visualization and Reporting
26mColumnar Storage: Redefining Data Management for the Modern Era
42mColumnar Storage: Redefining Data Management for the Modern Era
18mBuilding Data Lake with MariaDB ColumnStore
26mSoar Higher With MariaDB SkySQL - The Ultimate MariaDB Cloud
15mModern data platforms: AWS and MariaDB SkySQL
01h 00mColumnar DBMS: The Magic of Data Compression and Super-Fast Queries
43mThe Column's the limit