Topical Edition on Databases - Track 1

on YouTube ↗06h 54m

" summary

A concrete method for full-query incremental maintenance turns billion‑row SQL workflows into fast streaming updates: represent changes as Z‑sets (signed row counts), compile queries into a finite set of incremental operators, and run them as streaming circuits (delay/integrate primitives). One customer replaces a 60‑node Spark refresh on a 250 GB program with a single 15 GB machine doing 200 milliseconds refresh (roughly 5,000× less work); implementation is open‑source and integrated with Calcite.

AI-written summary · about

" chapters (11)

1. 0:00Conference welcome
2. 23:18Incremental computation in databases
3. 1:06:57Incremental computation for maintenance
4. 1:33:46Database changes with Z sets
5. 2:06:52Stream-based database computations
6. 2:33:03Traditional DBs struggle with updates
7. 3:13:41Cloud-native and AI databases keynote
8. 4:17:13ClickHouse optimization case study
9. 5:05:37Building distributed key-value store
10. 5:42:19Debugging HBase deadlock
11. 6:42:28Distributed DB design failures

" related talks

• 29mScalable Spreadsheet-driven End-User Applications with Incremental ComputationShachar Itzhaky, Sean Hadar · Onward! 2023
• 41mStreaming SQLJulian Hyde · Flink Forward Berlin 2016
• 40mOne SQL to Rule Them AllFabian Hueske · Flink Forward Europe 2019
• 41mUsing Databases to Pull Your Application WeightHarisankar P S · Southeast Ruby 2017
• 02h 00mTutorialAndres Freund, Marco Slot · PGConf Silicon Valley 2015
• 48mUnlocking Real-time Insights: Enhancing Your Databases With Stream ProcessingYingjun Wu · Big Data Conference Europe 2023
• 47mRethinking Relational Database CRUD Design Patterns Since We Live in a Big Data WorldChris Bohn · HighLoad++ 2017
• 50mChallenges that everyone struggles while productionizing Apache Spark workloadsChetan Khatri · ScalaIO 2019