NoSQL Python: making data frames work for you in a non-rectangular world
summary
Shows concrete techniques: use json_normalize and ijson to parse nested JSON and ragged CSV fields, subclass list-like column objects and swap class methods to make pandas hold arrays of dicts, and cluster per-actor time series with dynamic time warping for behavioral patterns. Demonstrations include Endomondo country clusters, Reddit top 25 thread trajectories, and nanoscale molecule signals. Notes performance: DTW is formally O(n^2) but often ~10% (empirically 1–10%) of n^2, and practical parsing uses jq and regex for cleaning.
AI-written summary · about
chapters (15)
- 0:00Introduction
- 1:04Why NoSQL matters
- 5:44Tidy data limitations
- 9:33Nested data challenges
- 12:05NoSQL from start
- 15:51NoSQL in real world
- 16:33NoSQL data into Python
- 18:38Custom Python classes
- 21:47Common NoSQL problems
- 23:12JSON parsing with JQ/I-JSON
- 26:01Clustering time series
- 30:55Time series analysis
- 31:38Future research
- 32:20Summary
- 32:42Q&A
related talks
32mDataFrames: scaling up and out
38mExpressive and fast dataframes in Python with polars
34mSmall Big Data: using NumPy and Pandas when your data...
44mHigh Perfomance Pandas
40mPython and pandas as back end to real-time data driven applications
01h 16mIntro to Data Analysis with Python Data-table
40mBeyond Pandas: The Great Python Dataframe Showdown
26mSmall Big Data: using NumPy and Pandas when your data doesn't fit ...