The Business Value of A/B TestingDoes it pay off to experiment versus shipping as fast as possible (without testing)?Oct 22Oct 22
Q&A on Sequential Statistics in A/B TestingSequential statistics are gathering interest and there are more and more questions posed by CROs looking into the matter. For this article…Aug 30, 2023Aug 30, 2023
Sequential Testing is About Improving Business ReturnsA central feature of sequential testing is the idea of stopping “early”, as in “earlier compared to an equivalent fixed-sample size test”…May 1, 2023May 1, 2023
False Positive Risk in A/B TestingHave you heard how there is a much greater probability than generally expected that a statistically significant test outcome is in fact a…Apr 19, 2023Apr 19, 2023
How to Run Shorter A/B Tests?Running shorter tests is key to improving the efficiency of experimentation as it translates to smaller direct losses from testing inferior…Dec 2, 2022Dec 2, 2022
Statistical Power, MDE, and Designing Statistical TestsOne topic has surfaced in my ten years of developing statistical tools, consulting, and participating in discussions and conversations with…Oct 25, 2022Oct 25, 2022
Published inTowards Data ScienceWhat Can Be Learned From 1,001 A/B Tests?A meta-analysis with insights into test duration, sample size, lift, power, confidence thresholds, and the performance of sequential tests.Oct 18, 20221Oct 18, 20221
Published inExperience StackTop Misconceptions About Scientific Rigor in A/B TestingIt is high time to dispense with the myth that “statistically rigorous A/B tests are impractical”.Jan 20, 2022Jan 20, 2022
Published inTowards Data ScienceImprove Your A/B Tests with 9 Lessons from the COVID-19 Vaccine TrialsWhat practitioners of online experimentation can learn and generalize from a complex processJan 12, 2022Jan 12, 2022
Published inTowards Data ScienceThe Perils of Using Google Analytics User Counts in A/B TestingInaccuracies in the user counts provided by Google Analytics can have disastrous effects on the validity of your statistical analyses.Oct 19, 2020Oct 19, 2020