Back to the Future
summary
Revisits old academic ideas like Prolog and logical programming and shows an application framework that composes logic blocks into pipelines. Demonstrates that a few dozen lines of configuration can generate production-scale big data jobs, describes the generator and its template mechanics, and reports actual deployments and throughput numbers for the generated pipelines. Emphasizes concrete code patterns, the config-to-artifact mapping, and where this approach fails or needs manual intervention.
AI-written summary · about
related talks
30mSMART DATA PIPELINES FOR AI
43mData Pipelines à la Mode
20mTime Travel and Provenance for Machine Learning Pipelines
48mTaming & Composing High Performance Stream-Oriented Processing Systems
29mDagster: a Framework for Data Processing Applications
16mData pipeline automation with built-in metadata and lineage (PRT054)
24mA Gentle Introduction to Building Data-Intensive Applications
26mGo with the Flow: Automating Your Workflows with Airflow