Data-Driven Engineering
summary
Argues for running automated assertions against production event logs (Avro logs ingested via Kafka into Hadoop/data warehouse) so tests exercise real scale and edge cases — Conversant runs 3–4 million requests/sec and terabytes/day. Shows SQL-style queries that sum deltas per SKU to tolerate eventual consistency and catch anomalies (example: black sweater return spike on April 15). Uses canary servers, user-based rollout, CI/CD with automatic rollback and concurrent experiment branches to run controlled experiments.
AI-written summary · about
chapters (21)
- 0:00Intro
- 1:23What is DataDriven Engineering
- 1:48DataDriven Product Management
- 3:13DataDriven Systems
- 8:44Testing in Production
- 9:08Unit Tests
- 12:14Test Functions
- 14:41Testing and Production
- 19:27Dont Test in Production
- 19:54Why Test in Production
- 26:17Conclusion
- 29:01Continuous Experimentation
- 30:28Return Rates
- 34:33Scientific Process
- 35:14Observational Study
- 36:04Controlled Experiment
- 41:34Software Development Lifecycle
- 42:50Differences
- 44:22Process
- 45:27Continuous Experiment
- 47:09Culture
related talks
42mDeciphering the way data is tested
45mChris Harbert: Testing AI at the Scale of Big Data
27mHow to be data-driven when you aren’t Netflix (or even if you are)
19mBig Data
34mData Engineering Architecture
37mEngineering data quality
26mDataOps: Accelerating Digital Transformation
42mTesting a Data Science Model