Data Contracts: Keep Your Weekend Work-Free!

on YouTube ↗20m

" summary

Data contracts are defined as enforceable agreements (not just tech) that specify schema, SLAs, interface and compliance requirements so dashboards and models stop generating 5 AM emergency alerts. The repo shows a literate data contracts pattern implemented in Quarto that embeds tests and human-readable ownership notes, and uses Git pull requests to negotiate changes and track responsibility. Existing tools like Great Expectations or Soda can run enforcement, but organizational ownership and minimal viable contracts are essential.

AI-written summary · about

" chapters (11)

1. 0:00Introduction
2. 1:49Trust in data outputs
3. 3:36Data reliability problems
4. 4:42Data contracts defined
5. 6:07Key roles in contracts
6. 7:35Elements of contracts
7. 10:48Collaborative creation
8. 12:13Technology for enforcement
9. 14:09Literate contracts
10. 16:18Overcoming resistance
11. 18:04Adopting data contracts

" related talks

• 32mData Contracts: Consensus as CodeRyan Collingwood · PyCon AU 2023
• 26mData contracts: driving data culture changeAndrew Jones · Data Teams Summit 2023
• 27mLive in the Data Wild West: The Data Contracts SheriffTal Peretz · hayaData 2024
• 28mDataEngBytes 2023 - SYD-T1-03 - Chad SandersonChad Sanderson · DataEngBytes Sydney 2023
• 35mPanel DiscussionRyan Boyd, Chad Sanderson, Joe Reis, Emily Melhuish · DataEngBytes Melbourne 2023
• 41mDataEngBytes 2023 - BRI-T1-01 - Chad SandersonChad Sanderson · DataEngBytes Brisbane 2023
• 32mExtreme Contracts: the Freedom to Write the Tests you NeedJacopo Romei · TestCon Europe 2019
• 31mData Reliability Engineering: Secret Of Our SuccessAkshay Karle, Carmen Mardiros · SREday 2023 London