Confident Data Migration: automatic regression testing
summary
ContentSquare migrates Redshift → Elasticsearch → ClickHouse while traffic grows tenfold and avoids overnight outages by replay production load to legacy and new backends, storing raw JSON responses in a comparator DB and measuring relative errors (quantiles, max). Two comparator strategies run: focus comparators (fix all but one dimension to isolate metric bugs) and prediction-request comparators (replay real logged queries for realistic coverage). A phased rollout catches regressions like a faulty activity-rate computation; iterate, fix, rerun, then finalize.
AI-written summary · about
chapters (10)
- 0:00DATAXDAY
- 3:26Infinite request possibilities
- 6:18Comparators: overview
- 7:43Comparators: individual result
- 8:24Comparators: aggregated result
- 8:52Which requests to play?
- 9:29Focused comparators
- 11:18Prod Request Comparator
- 13:08Narrow down the problem and fix it!
- 14:24What if the legacy system is wrong?
related talks
27mModernize Your Data Warehouse
42mTransform your data approach: Develop a modern data strategy
31mFrom Raw Data to Real Impact: Lessons from Real-World Transformations
01h 03mDiscovering the value of a cloud data platform
28mModernize Your Data Warehouse
32mMove to managed analytics
26mEffective Data Migration: Challenges & Success
40mData Migration Lessons from the Trenches