Designing Feature DSLs: Principles and Tradeoffs
summary
A bespoke DSL called scowl DSL expresses streaming and on‑demand feature transforms as composable components rather than opaque batch jobs. It implements a compiler, syntax highlighting and a REPL and compiler so expressions run against real data, supports stateful streaming join patterns (example: last_login → minutes_since), type inference, lists/structs/JSON, missing values and embeddings, and treats the whole pipeline as a DAG for feature versioning and offline/online consistency. The implementation is in production for low‑latency fraud/security use cases.
AI-written summary · about
chapters (4)
related talks
41mHow to Elm-ify Your ML
22mA Type System For Feature Engineering (Extended Abstract)
21mFinally Tagless DSLs and MTL
33mScala DSLs and Probabilistic Programming
39mDesigning 1000 genomes in a single bound: The Codon DSL
20mSemblance: Feature Generation In Real-Time and Batch Without Time-Travel
31mF3: A Compiler For Feature Engineering
36mDeclarative Feature Engineering