~/talk

DataEngBytes 2023 - SYD-T3-04 - Abhinav Goyal

on YouTube 21m
Talkat DataEngBytes Sydney 2023 · on Friday, 2023-08-25· [frontend-architecture, data-engineering]

speaker

name:Abhinav Goyal
links:none on file

resources

video:https://youtu.be/iWGfyfQ-Rxg
slides:none on file

summary

Argues that a compiler-style semantic layer turns brittle pipelines into verifiable programs: ban base types and replace ints/strings with semantic types with units and patterns (emailID, ABN, length, currency) so the system auto-generates converters, enforces contracts, and catches unit/format bugs before runtime. Shows mock code and examples that produce auto-generated workflow sequencing and parallelism from metadata, claims this collapses the 250+ tool surface and enables data quality checks, contract enforcement, and safer schema evolution across the stack (Ordinatim is building toward this).

AI-written summary · about

chapters (10)

  1. 0:00Self-driving companies
  2. 1:07Business needs from data science
  3. 3:55Data engineering reality
  4. 6:04Domain knowledge gap cause
  5. 7:29Ignoring domain knowledge costs
  6. 9:58Failed gap-bridging attempts
  7. 12:04Systems with semantic types
  8. 16:16Semantic types for business logic
  9. 17:23Workflow tracking and parallelism
  10. 19:54Data and product roles future

related talks

~/search
type to search 6,249 conferences, 138,987 speakers and 185,172 talks
~/help
cmdctrl k
search every conference, speaker and series
/
focus the filter search
h
home
c
conferences
s
speakers
e
series
t
light / dark — the status bar button does the same
?
this help
esc
close this, then clear every filter — or just the search, while you are typing in it

Ignored while you are typing. Pages without a filter rail have no search to focus.