Numerous times our clients and customers ask: how many visitors do I need to test in order to get statistically significant results in an A/B test? Unfortunately, there is no straightforward answer to this question as it depends on a number of factors. In the past, we have provided you with an A/B test duration calculator in Excel and also provided with an online tool to calculate the same. However, how it is calculated is sort of a black magic.

In following presentation, I describe just what exactly happens when we calculate number of visitors needed for an A/B test. As the presentation aims to be a high level overview, I won’t be delving into mathematics of it.

Hope you like the presentation! Please let me know if you have any questions or need clarifications.

Related posts:

1. A/B test duration calculator (Excel spreadsheet)
2. What you really need to know about mathematics of A/B split testing
3. How reliable are your split test results?
4. A/B testing tricks: How to test additional (new) content in a split test?
5. How to A/B test AdSense revenue using custom channels

Tags: ab test duration, multivariate test duration, split test duration
Posted in A/B Split Testing, How To on January 18th, 2012 | No Comments »

No Comments »

No comments yet.

RSS feed for comments on this post. TrackBack URL

Leave a comment