FlashProfile - A Framework for Synthesizing Data Profiles
summary
A framework called FlashProfile uses the PROSE inductive-synthesis engine (PROSE) to cluster strings by a pattern-aware similarity (hierarchical clustering guided by a cost function) and synthesize regex-like profiles over an extensible atom language that includes constants, character classes, regex fragments and functions. It profiles 75 large real datasets across 153 tasks with median profiling time ~0.7s, supports interactive refinement and custom atoms, and reduces examples needed by Flash Fill for mixed-format inputs.
AI-written summary · about
chapters (14)
- 0:00Intro
- 0:39The Challenges of "Big" Data
- 2:05State of The Art
- 4:22Syntactic Profiling
- 5:56Key Challenges
- 7:15Main Contributions
- 8:23Overview of FLASH PROFILE
- 9:27Profiling via Clustering
- 11:16Pattern Synthesis
- 12:45Profile-Guided Interaction for PBE
- 14:14Quality of Generated Profiles
- 15:21End-to-End Profiling Performance
- 16:43Conclusion & Future Work
- 17:46Publicly-Available Artifacts
related talks
17mGuiding Dynamic Programing via Structural Probability for Accelerating Programming by Example
31mProfiling Formats and Why You Should Care
17mModeling Black-Box Components with Probabilistic Synthesis
16mScalable Synthesis of Regular Expressions From Only Positive Examples
38mProgram Synthesis: A Dream Realized?
14mEfficient Foundation Models for the Agentic Era
16mJust-in-Time Learning for Bottom-Up Enumerative Synthesis
20mSemblance: Feature Generation In Real-Time and Batch Without Time-Travel