Build your own data warehouse for personal analytics with SQLite and Datasette
summary
Concrete takeaways: use SQLite (single-file, ubiquitous, example photos.sqlite at ~856 MB) plus Datasette (web UI, editable SQL, JSON/CSV export, ~30 plugins) to ingest real sources — Twitter, GitHub, Apple Photos, Apple Health, Google Takeout, Foursquare, 23andMe — and build a personal warehouse. Examples include a 19,000-row public-toilets map, 1.5M MPs gift rows, a 600,000-row 23andMe import, interactive charts and maps, and a working personal search engine Dog Sheep Beta that ties ~193,000 items together.
AI-written summary · about
chapters (5)
- 0:00Start
- 3:21Dataset Facets
- 4:52Sqlite
- 18:45Alternative to Cron
- 19:30Do I Plan To Steal Palantirs Market Share
related talks
02h 46mData analysis with SQLite and Python
15mDatasette and Datasette Publish
27mUsing embedded SQL engines for plotting massive datasets on a laptop
23mFANTASTIC DATABASES AND WHERE TO FIND THEM
29mHow to make public data more accessible with "baked" data and DuckDB
47mFun with BIG Data with various databases
58mA Scientist's Guide to the Relational Data Model
38mBuilding a Memex (with Ruby!)