UDF to SQL Translation through Compositional Lazy Inductive Synthesis
summary
A tool called CLIS implements compositional lazy inductive synthesis to convert Scala UDFs into pure SQL by building a data‑flow graph, applying DFG decomposition and node-wise CEGIS synthesis, and coarsening failed nodes until recomposition. Evaluated on 100 GitHub Spark UDFs, 63 are manually judged SQL‑expressible and CLIS synthesizes 58 of those within one hour (54 within five minutes, median 9s), yielding per‑UDF speedups (geometric mean ~3.5, many >1.3×) and outperforming rule‑based Froid and a reimplemented Casper baseline.
AI-written summary · about
chapters (9)
- 0:00Introduction
- 0:39UDFs
- 1:17Why UDFs
- 1:46UDF to SQL Translation
- 3:02Inductive Synthesis
- 5:55CGS
- 8:18Running Example
- 9:29Summary
- 10:31Evaluation
related talks
47mApache Spark Lakehouse Pushdowns with Transpilation
14mAutomated Translation of Functional Big Data Queries to SQL
27mFunctional Programming on Top of SQL Engines
48mCompositional, Expressive, Performant Database Queries: Choose 3
16mA Compiler for Fused Relational Operations on Multisets
26mRun Numba functions in SQLite: WTF?
30mML meets NoSQL: Integrating Python user-defined functions (UDFs) with N1QL for analytics
41mHow types can turn a SQL interpreter into a SQL compiler