Lineage Quality Measurement
summary
Meta builds a Lineage Quality Measurement framework that quantifies lineage tools by recall and precision metrics, using sampled ground truth from the company’s privacy review ground truth and random samples of tool outputs that receive expert labeling. Concrete examples show one tool with 100% recall but 67% precision and another with 50% recall and 100% precision. The framework lets teams compare tools, identify complementary pairings, and prioritize engineering to reduce the biggest remaining risks.
AI-written summary · about
chapters (11)
- 0:00Lineage quality measurement intro
- 1:04What is lineage and challenges
- 3:12Lineage tools trade-offs
- 5:44Problems with multiple tools
- 6:48Lineage quality framework
- 7:29Measuring recall
- 8:56Estimating recall with ground truth
- 9:59Measuring precision
- 11:04Precision sampling and review
- 11:48Metrics to improve tools
- 12:10Lessons learned and future
related talks
15mArchitecting Scalable Data Lineage Graph for Privacy Compliance and Agentic Analysis
12mUsing GenAI to Accelerate Privacy Implementations
51mInnovate, Validate, Elevate: LabVIEW Quality Paradigm
24mThe 3-pillar, balanced quality model of Proactive QA, Detective QA and Reactive QA
12mConvergent Quality Engineering
14mEssential Metrics for the Quality Assurance
36mHow Good Is Your Software Product? (Evolving "Quality"Throughout The SDLC)
41mCode Metrics Q&A