When metrics lie
summary
Pinterest builds a certification process for core metrics, starting with pin impression = visible ≥1s and minimum viewport fraction, writing product specs, server-side deduping, platform audits and regression UI tests that run on Firebase Test Lab + Flank. Certification takes weeks–months; focus on the 3 surfaces that drive 90% of impressions. Results: caught 5 regressions in October, fixed a Chrome intersection observer bug, and reduced platform mismatches (one case reported 21% iOS overcount).
AI-written summary · about
chapters (11)
- 0:00Introduction
- 1:23Why metrics lie
- 7:45Pinterest’s certification process
- 10:55Defining pin impressions
- 15:26Data team’s role in validation
- 18:16Client-side implementation
- 21:07Testing for metric accuracy
- 24:41Communicating certified metrics
- 28:34Does certification work?
- 30:24Tips for small teams
- 31:27Q&A
related talks
12mEngineering Productivity, Infrastructure
52mAB-tests, lies, damned lies, and statistics.
20mReal metrics for front-end performance
29mNumbers, because metrics matter
24mVanity performance metrics are worthless
28mMeasurement and Analysis for Sustainable Service Delivery
24mMetrics-driven development
58mWin with Metrics: How to Track and Show the Value of Your Program