Real-time A/B: how to get metrics for experiments
summary
Describes building a real-time A/B metrics counting system for high load and large data volumes, explaining why real-time experiment metrics are needed, the system architecture under the hood, and the operational challenges encountered.
AI-written summary · about
chapters (10)
- 0:00Introduction to real-time A/B
- 3:23Need for fast metrics
- 8:30System architecture
- 12:12Simple metric calculation
- 13:01Data sharding issues
- 19:54Complex metrics processing
- 25:08Fraud and session handling
- 28:01Results and deployment
- 29:07Conclusions
- 30:32Q&A
related talks
47m10K Metrics, 500 A/B Experiments and 500M p‑values Every Day — How Is That Possible?
47mAccounting for user behavior data in A/B testing metrics
43mA/B Testing: What to Pay Attention To
24mA B Platform Interactive Analysis in Realtime
55mHow to Improve Your Game's Retention & Monetization Through Data-Driven Experimentation
39mUnder the hood of fast traffic splitting for A/B testing
19mA/B User Testing When Your CDN Cache Messes With Your Results
28mMeasurement and Analysis for Sustainable Service Delivery