The statistics of A/B testing results can be confusing unless you know exact formulas. Earlier, we had published articles related to mathematics of A/B testing and also have a free A/B testing calculator on the site to see if your results are significant or not. The articles provides an introduction and calculator simply provides an interface; the real formulas used for calculate statistical significance of split testing results are still missing.
Excel sheet with A/B testing formulas
So, we have come up with a FREE spreadsheet which details how exactly the significance is calculated. You just need to provide e thnumber of visitors and conversions for control and variations. The spreadsheet will automatically calculate for you significance, p-value, z-value and other relevant metrics for any kind of split testing (including Adwords). Of course, you can see the relevant formulas in the spreadsheet. Click the screenshot below to download the calculator (spreadsheet):
Please feel free to share the file with your friends and colleagues or post it on your blog / twitter.
PS: By the way, if you want to do quick calculations, we have a version of this calculator hosted on Google Docs (please make a copy of the Google Doc sheet into your own account before you make any changes to it).
Update: Thanks to Jai (in the comments below), we had noticed a minor error in conversion rate range calculations (though significance results were unaffected). The error in fixed in the latest version of spreadsheeet.
Stay up to date
- @davidshawblog David, what happened? #
- Improved Visual Hierarchy Increases Online Sales by 35.6% for @UnderwaterAudio http://t.co/b5IR2YJJ27 #
- @xeniusmedia No, not right now. However, this is a very interesting suggestion! We will add it on our roadmap. #
- @plindelauf What's your location? Can you tell which website do you see this on? Can you send your ping details to our servers? #
- How Images Can Boost Your Conversion Rate http://t.co/m1sgA1cJVZ #
Write for this blog!
We accept high quality articles related to A/B testing and online marketing. Send your proposals to email@example.com