Roche's Maxim of Data Transformation - By Example
summary
Roche’s Maxim says transform as far upstream as possible and as far downstream as necessary and the talk gives concrete rules: do simple joins and string concatenation in SQL or a shared Power Query view, use DAX only for slicer‑dependent logic, shred JSON before downstream hy‑ing when you must track the newest row, avoid pulling a REST API 40 million rows at 50 rows per call, prefer PolyBase or Fabric one‑lake shortcuts when access or skills make changing the source impractical.
AI-written summary · about
chapters (10)
- 0:00Data engineering challenges
- 4:09Introduction to the maxim
- 5:50Data sources and types
- 13:11Tools for transformation
- 17:24Understanding the maxim
- 22:20Upstream vs downstream
- 25:26Drivers of transformation
- 30:27Practical examples
- 37:15Complex scenarios
- 43:14Conclusion
related talks
47mHow to pick the best Data transformation strategy
56mData Unleashed: A Developer's Perspective on Navigating the Architecture Maze
01h 13mDiscover, Acquire, and Transform Your Data with Power Query
33mMaximizing Data Usability for Impact
31mData Unleashed: A Developer's Perspective on Navigating the Architecture Maze
27m"Mind, Body, Heart & Soul" - Data Mesh implementation journey at Roche Diagnostics
51mPower BI Composite and Hybrid Models
32mDataEngBytes 2023 - BRI-T1-10 Rishu Saxena & John Cosgrove