DataEngBytes Melbourne 2023
who it's for
data engineers, analytics engineers, machine learning engineers, data architects, platform engineers, data product managers, site reliability engineers
topics
conceptual and logical data modelling; data mesh implementation patterns; schema-on-read JSON ingestion; Snowflake JSON extraction; Postgres PostGIS geospatial queries; DuckDB browser WASM execution; Apache Flink streaming with Kafka; feature stores for training and inference; declarative data quality checks and automated lineage; event-time watermarks and fault-tolerant checkpoints
| title | author | length |
|---|---|---|
| DataEngBytes 2023 - MEL-T1-01 - Joe Reis | Joe Reis | 26m |
Argues that DATA MODELLING IS DEAD! LONG LIVE DATA MODELLING! but actually data modelling is critical for reliable AI, citing concrete problems like reactive… | ||
| DataEngBytes 2023 - MEL-T1-02 - Kendra Vant | Kendra Vant | 33m |
Argues that successful AI products rest on clarified data practice and cross-team literacy, citing seven years shipping AI products, prior Enterprise data work… | ||
| DataEngBytes 2023 - MEL-T1-03 - Adric Streatfeild | Adric Streatfeild | 30m |
ANZ builds a cloud data mesh that powers the ANZ Plus app (eg. expense prediction) and claims the first four capabilities cut onboarding from five months to… | ||
| DataEngBytes 2023 - MEL-T1-04 - Michael Hyatt | Michael Hyatt | 19m |
Demonstrates schema-on-read by immediately storing API payloads (two JSON endpoints and one XML) as raw strings/JSON plus a timestamp, then parsing later with… | ||
| DataEngBytes 2023 - MEL-T1-05 - Izzudin Hafiz | Izzudin Hafiz | 24m |
Claims that data teams commonly rely on Excel as primary catalog and that most teams cannot confidently answer "what data do we have" leading to lost trust… | ||
| DataEngBytes 2023 - MEL-T1-06 - Lizzie Macneill | Lizzie Macneill | 25m |
A live map server built on Postgres + PostGIS loads NSW open-data EV chargers, converts coordinates with ST_Transform, and serves them as GeoJSON to a Leaflet… | ||
| DataEngBytes 2023 - MEL-T1-07 - Ryan Boyd | Ryan Boyd | 27m |
Argument for replacing monolithic cloud warehouses with a hybrid local+cloud model that actually runs: a 20‑MB in‑process SQL OLAP engine (DuckDB) embeds in… | ||
| DataEngBytes 2023 - MEL-T1-08 - Troy Sellers | Troy Sellers | 27m |
Shows an implemented anomaly detector using Apache Flink and Kafka running on Aiven managed services that flags IoT devices whose CPU usage averages above 80%… | ||
| Panel Discussion | Emily Melhuish, Joe Reis, Ryan Boyd, Chad Sanderson | 35m |
Debate over when to add guardrails gives concrete rules: treat data contracts as codified APIs (schema, SLAs, ownership) when data is a product; delay heavy… | ||
| DataEngBytes 2023 - MEL-T1-10 - Josh Devlin | Josh Devlin | 20m |
Argues that Zhamak Dehghani’s Data Mesh only solves large enterprise scale problems (think thousands of employees) and lists five concrete failure modes… | ||
| DataEngBytes 2023 - MEL-T1-11 - Emily Melhuish | Emily Melhuish | 28m |
Builds datasets from raw sources: parses kHz engine sensor logs on‑prem with vanilla Python, stores in Postgres/TimescaleDB and computes flight parameters used… | ||
| DataEngBytes 2023 - MEL-T2-01 - Mikiko Bazeley | Mikiko Bazeley | 30m |
Claims that 80% of a data scientist’s time is spent on data and that 1 in 5 companies build a feature store drive the talk, backed by concrete patterns… | ||
| DataEngBytes 2023 - MEL-T2-02 - Vikas Rajput | Vikas Rajput | 24m |
Data engineering shifts from slow, centralized conformance toward fast, iterative AI-assisted loops that cut months of dev time to days by handing routine code… | ||
| DataEngBytes 2023 - MEL-T2-03 - Joshua Yu | Joshua Yu | 24m |
Claims that combining a large language model with a knowledge graph makes generative AI up-to-date, controllable and explainable. Implements embeddings… | ||
| DataEngBytes 2023 - MEL-T2-05 - Paul Brebner | Paul Brebner | 26m |
A simulated drone-delivery system uses Cadence workflows for stateful drone and order logic, connects those workflows to Apache Kafka topics for event… | ||
| Panel Discussion | Joe Reis, Mikiko Bazeley | 31m |
Panelists argue that tooling is transient so prioritize the data and model life cycle over specific libraries; treat LLM deployments as ML problems with… | ||
| DataEngBytes 2023 - MEL-T3-01 - Rajath Vanikul | Rajath Vanikul | 23m |
Binary classifier detects electric vehicles from smart‑meter consumption by reducing four weeks of time series into monthly customer‑level snapshots and… | ||
| DataEngBytes 2023 - MEL-T3-02 - Alexei de Lauw | Alexei de Lauw | 18m |
Demonstrates logging and tracing built on Snowflake event table primitives to solve three concrete bottlenecks: debugging long stored procedures and UDFs… | ||
| DataEngBytes 2023 - MEL-T3-03 - Akira Wang & Abdi Farah | Akira Wang, Abdi Farah | 24m |
Metadata implementation uses Airflow, Databricks, AWS Glue and system views from Redshift Snowflake and S3 to ingest lineage and job stats into a Postgres… | ||
| DataEngBytes 2023 - MEL-T3-04 - Sarah Young | Sarah Young | 29m |
Explains a hardware root exploit in the Tegra X1 that uses shorting joy‑con pins to force a recovery boot, then a buffer overflow overwrites signature check… | ||
| DataEngBytes 2023 - MEL-T1-05 - Fabio Ramos | Fabio Ramos | 21m |
Apache Iceberg is presented as an open table format that brings ACID transactions, time travel and hidden partitioning to object storage, using data files plus… | ||
| DataEngBytes 2023 - MEL-T3-06 - David Colls | David Colls | 31m |
Maps seven specific Lean wastes onto data production, shows how value‑stream mapping and data as a product stop overproduction and inventory, and demonstrates… | ||
| nothing matches | ||