Crunch Conference 2021
2021-10-07..2021-10-08
Budapest, Hungary
[Crunch Conference 2019, Crunch Conference 2021, Crunch Conference 2022]
who it's for
data engineers, data scientists, machine learning engineers, analytics engineers, data platform engineers, mlops engineers, data product managers
topics
explainable boosting machines (EBMs); differentially private models (DP-EBMs); data lineage and discovery; dbt v1.0 metadata artifacts; Iceberg/Delta/Hudi lakehouse engines; feature stores for production ML; Apache Pinot realtime analytics; in-memory row and column store (SingleStore); Vizzu C++->WASM animation engine; dataset engineering for CNNs; model repository and Kubernetes serving; open data validation and reformatting
| title | author | length |
|---|---|---|
| INTERPRETABLE AND DIFFERENTIALLY PRIVATE MACHINE LEARNING | Rich Caruana | 01h 01m |
Claims that EBMs (Explainable Boosting Machines) reach boosted-tree/NAM accuracy on tabular data (average ROC ~0.866 vs ~0.86) while remaining fully… | ||
| BACK TO THE FUTURE: EMERGING TRENDS IN DATA ENGINEERING | Ananth Packkildurai | 57m |
Seven concrete trends claim to reshape data work: prioritize start with lineage — build data lineage and discovery first when creating a new warehouse; expect… | ||
| KEEPING UP IN A DATA-INTENSIVE WORLD WITH SINGLESTORE | Jordan Tigani | 46m |
SingleStore claims a single engine for transactions, analytics and full‑text search by combining an in‑memory row store with an SSD transaction log, a seekable… | ||
| MY AWFUL EX-DATASYSTEMS | Jeffrey Theobald | 44m |
Covers concrete failures and fixes: Flume crashes on 4‑byte UTF‑8 emojis, MongoDB limits and alternatives, Spark hype versus real needs, and a legacy pipeline… | ||
| BREAKING YOUR DATA TEAM OUT OF THE SERVICE TRAP | Emilie Schario | 39m |
Argues that data teams move from reactive service desks to data products by building plumbing then metrics then insights; Netlify uses Snowflake, dbt (core and… | ||
| FINDING BAD BEES IN THE HIVE | András István Nagy | 20m |
Platform processes tens of millions of devices and reduces network usage into hourly rates (168‑dim week) with only 2–20 observations per day per device… | ||
| TOWARDS THE GRAMMAR OF ANIMATION | Peter Vidos | 35m |
Claims a reusable animation engine that morphs between chart types and ships as an open-source Vizzu JS library (public beta). Engine follows a… | ||
| THE STATE OF MLOPS - MACHINE LEARNING IN PRODUCTION AT ENTERPRISE SCALE | Bas Geerdink | 41m |
Argues MLOps is DevOps applied to ML, with concrete architecture: Airflow training jobs produce serialized models stored in a model repository + Kubernetes… | ||
| USER-FACING, REALTIME ANALYTICS USING APACHE PINOT | Neha Pawar | 44m |
Apache Pinot is a distributed OLAP store that ingests from Kafka Kinesis or Pulsar and serves queries from in-memory segments then flushes to deep storage… | ||
| THE ROAD TO DBT V1.0 | Jeremy Cohen | 42m |
dbt Core moves from 0.x to 1.0 by locking down metadata artifact contracts, splitting adapters and plugins into separate repos for better maintenance, and… | ||
| RIGHT DATA AT THE RIGHT TIME: FUELING CNN DEVELOPMENT WITH DATASET ENGINEERING | Ridzi Péter | 36m |
Continental ADAS builds an automated dataset pipeline that stages GPS/timestamp enrichments, runs CNN pre-labeling on every second of recordings, stores… | ||
| OPEN DATA - THE NEW GOLD WITHOUT THE RUSH | Dániel Antal | 37m |
Open data often arrives as messy legacy files that require more labour than buying commercial feeds, and reprocessing costs can exceed market alternatives; the… | ||
| OPERATIONALIZE THE DATA CATALOG | Grant Seward | 34m |
Argues data catalogs are too reactive and shows a working approach that makes them proactive: combine SQL log parsing with a synchronous Amundsen API hooked… | ||
| Modelling and predicting customer lifetime value | Egor Kraev | 34m |
Compares classic Buy Till You Die models implemented with the lifetimes Python package against a custom variational autoencoder + RNN that ingests event… | ||
| Right data at the right time: Tools demo | Csongor Szántó | 16m |
Demo of a tool called Cider that manages deep‑learning image labels, stores pixelwise semantic segmentations, and exposes a metadata query UI that returns… | ||
| How Machine Learning and AI Can Support the Fight Against COVID-19 | Dmitry Soshnikov, Francesca Lazzeri | 57m |
Uses the CORD corpus of ~400,000 papers (≈200,000 full text) and processes documents in 500‑paper chunks, parallelizing via Azure ML sweep jobs to avoid 11–12… | ||
| nothing matches | ||