DataEngBytes 2023 - SYD-T1-03 - Chad Sanderson

on YouTube ↗28m

" summary

Defines data contracts as API-like agreements that declare schema semantics and SLAs and enforces them programmatically via YAML contract per resource with a one-to-one mapping and consumer-proposed ownership, tiered actions (0–2) and CI CD checks that either block or notify changes. Implementation uses GitHub PR workflows to show exactly which fields break, integrates with catalogs like Amundsen indexing, and ties monitors into the contract platform. Deployed ~15 times, spoken with ~800 companies, and typically cuts data quality incidents by about 50% immediately.

AI-written summary · about

" chapters (10)

1. 0:00Introduction
2. 1:25AI impact on data quality
3. 5:42Why data quality is low
4. 8:09Organizational causes of issues
5. 12:24Monitoring and governance limits
6. 14:55Data contracts explained
7. 20:14Implementing data contracts
8. 23:46Contracts with data catalogs
9. 25:10Changing culture with contracts
10. 27:20Summary and Q&A

" related talks

• 41mDataEngBytes 2023 - BRI-T1-01 - Chad SandersonChad Sanderson · DataEngBytes Brisbane 2023
• 30mDataEngBytes 2023 - PER-T1-01 - Chad SandersonChad Sanderson · DataEngBytes Perth 2023
• 27mLive in the Data Wild West: The Data Contracts SheriffTal Peretz · hayaData 2024
• 20mDataEngBytes 2023 PER-T1-10 - Matthew CarterMatthew Carter · DataEngBytes Perth 2023
• 36mDataEngBytes 2023 - PER-T1-02 - Josh ArmitageJosh Armitage · DataEngBytes Perth 2023
• 20mData Contracts: Keep Your Weekend Work-Free!Nick Pelikan · posit::conf(2024)
• 30mDataEngBytes 2023 - MEL-T1-03 - Adric StreatfeildAdric Streatfeild · DataEngBytes Melbourne 2023
• 26mData contracts: driving data culture changeAndrew Jones · Data Teams Summit 2023