DBTB INT Sandy Ryza r
summary
Demonstrates a Spark-based time series library that focuses on concrete layout choices rather than algorithms, arguing that contiguous per-entity layout and time-bucket grouping determine query latency and memory use; shows how grouping rows by entity and chunking time windows enables fast aggregations and joins in Spark, describes the practical engineering tradeoffs made when building the library, and reports that the layout-first approach substantially improves query speed and scalability.
AI-written summary · about
related talks
29mDataEngBytes 2023 - BRI-T1-03 - Yousry Mohamed
07mTime Series Databases
40mYTsaurus SPYT: Helping the Apache Spark Scheduler Be Even More Efficient
24mImproving interactive querying experience on Spark SQL
55mSpark Architecture
50mIntroduction to Real-Time Big Data with Apache Spark
36mIn-memory data pipeline and warehouse at scale
55mThe ELT pattern in action with DBT and SQLMesh