Pragmatic Code Generation for Efficient Execution
summary
NetSpring builds a push-based, vectorized interface that generates small, per-operator C++ fragments compiled with LLVM rather than one fused query binary. Code fragments are parameterized to reuse cached modules and precompiled runtime pieces are linked in, yielding faster compile times and a reported ~75% compile-cache hit on TPC-DS runs. Operator boundaries remain for precise metrics, packets (batches) enable cache-friendly memory access, and LLVM JIT event listeners plus symbol tables let gdb/perf produce symbolized stacks and profiles for debugging and profiling.
AI-written summary · about
chapters (20)
- 0:00Start
- 3:39AP: Converged Analytics Platform
- 8:02Volcano of Operators
- 14:23Cost of Abstraction: Dynamic Dispatch
- 16:11erformance: select sum(revenue) from txn
- 17:09Code Generation & Vectorization
- 22:14perator Interface
- 26:16JetSpring Execution Engine
- 27:35Faster Execution
- 29:20Faster Compilation
- 30:04Compile Cache Hit Rate
- 31:42Intuitive Algorithms
- 33:57Easier Instrumentation
- 35:26ractical Considerations For Generating Code
- 35:38Compilation Cost: LLVM Module Caching
- 36:58Compilation Cost: Secondary Compile Cache
- 37:38Compilation Cost: Parameterized Constants
- 41:11Difficult to Debug: Symbolize Crash Stacktrace
- 43:53ain to Profile: Symbolized Gperf Profiles
- 46:24ummary
related talks
47mHacking Postgres Executor For Performance
48mPart I: A Performance Analysis of a Trading System over Compliers
52mWriting fast C code for a modern CPU (and applying it to PostgreSQL)
26mFast Erasure Coding for Data Storage: A Comprehensive Study of the Acceleration...
28mSpEQ: Translation of Sparse Codes using Equivalences
16mA Compiler for Fused Relational Operations on Multisets
17mBring Your Own Data Structures to Datalog
32mHacking Postgres Executor For Performance