Sequential Testing Simplified with Basemath
summary
GetYourGuide reports 40% reduction in experiment timelines by using Basemath, a Python package (pip install basemath-analysis) that implements sequential testing with a binding futility boundary, a one-sided test, support for streaming and group (batched) sequential checks, binary or continuous metrics, and a user-set maximum run time. Basemath avoids the usual repeated numerical integration by redistributing error across paths (a probabilistic “coin flip” at batch checkpoints), speeds up evaluating many concurrent experiments, and outputs intercept, slope (MDU) and stop/continue decisions.
AI-written summary · about
chapters (6)
- 0:00Introduction to Basemath
- 6:43Limitations of fixed-horizon testing
- 9:16Why sequential testing helps
- 16:25Demo of Basemath package
- 22:32Implementation and rollout process
- 28:16Q&A session
related talks
47mexpectation: A modern take on statistical A/B testing with e-values and martingales
28mDerivations, Not Just Simulations: Teaching Applied Mathematics with Scientific Python
29mIntuitive A/B Test Evaluations for Coders
28mThe Hitchhiker's Guide to Advanced A/B Testing Techniques
26mAnti-Patterns in A/B Testing - or How does AB Testing Scale?
18mHow Data Scientists Broke A/B Testing (And How We Can Fix It)
32mExpAn: A Python library for advanced statistical analysis ...
01h 30mBayBE: A Bayesian Back End for Experimental Planning in the Low-To-No-Data Regime