Trusting Your Data Science Pipelines: A Quantitative Approach
summary
Two Sigma describes building a Quantitative Reliability Engineering team that finds concrete failures and fixes: a Java 7 to Java 7 to Java 8 upgrade induces non‑determinism from hashmap iteration order plus floating‑point non‑associativity; a JIT optimization makes math.pow return differing results across runs; Monte Carlo pipelines break from hard‑coded seeds and stateful models. Remedies include switching to order‑guaranteed maps (LinkedHashMap/TreeMap), making models stateless, adding sensitivity tests, defining numeric contracts, and shipping reproducibility frameworks that measurably reduce flaky outputs.
AI-written summary · about
chapters (4)
related talks
49mShifting Safety Techniques To a Statistical World
32mReliable Machine Learning
35mSecure Numerical Computing is Hard: Lessons from 10 Years of Open Data Science & the Long Road Ahead
28mEngineering Data Reliably Using SLO Theory
05mData Quality and Validation in ML Pipelines
30mFrom Black Box to a Known Quantity: How to Build Predictable, Reliable ML-based
48mChaos Engineering: The Path to Reliability
38mData Secrets from a Platform Engineer