Challenges of database testing
summary
Claims a practical pipeline: sophisticated logging ties activity IDs to rows, CDC and ETL handle millions of heterogeneous records, and four predictive statistical algorithms live as a 600-line algorithm in the database. Builds an HTML report plus Excel-driven checks that let business verify outputs. Daily health-check validations run overnight (~4 hours); regression-style reporting flags post-release failures; one result: 40,000 rows in 15 minutes validated end-to-end.
AI-written summary · about
chapters (5)
- 0:00Start
- 1:11View of a complex business logic intensive database application
- 4:54The challenge
- 5:25Effectively monitoring database activities
- 14:55Tasks of a DB QA
related talks
30mHow to Test BigData/ETL Solutions?
39mModern Data Pipelines Testing Techniques: A Visual Guide
39mWhat Can Go Wrong With Testing an ETL Pipeline?
25mTesting Data Pipelines
37mData Validation at Scale: Managing Data Quality in Complex Data Pipelines
10mBuilding High-Performance Financial Data Pipelines
42mAutomation Testing in a Real Big Data Project
47mEfficiency up, failed releases down; compliant production data for your test and dev environments