Thank you for providing this great A/B test duration calculator!

I am also interested in getting the duration estimates when the chance of finding a difference is 90% or 99%.

I found the 90% duration estimate by using Table 2.1 in the book you reference on this page, i.e in the chapter from “Statistical Rules of Thumb” at http://www.vanbelle.org/chapters/webchapter2.pdf (the two sample numerator is 21). And I used the formula in your spreadsheet to add a few columns to display the 90% values.

Could you please tell me what the missing row with Power = 0.99 from Table 2.1 is, or point me to a reference that has this information.

Better yet, it would be great to add these extra 2 “confidence levels” to your calculator!

Thanks,

Lori Proos

I want to separate cents from dollars (or in my case euros), like _vis_opt_revenue=49.05. Or do I need to round the number?

Our customers have been quite successful with sliders, however as you state, testing is the key since this should never be assumed.

Considering the usability aspect, for me, I find if a taboo to auto-start a video, and from the test in this post, it looks like it was to be triggered on click. Another great thing about a slider is that the standard is for them to play automatically on page load, thus stepping the viewer through, with no specific interaction required.

Such reversals/crossovers of test results are common and too be expected, but it doesn’t necessarily mean the slider option was the best option.

Depending on your product/service, campaign or whatever, your best, hottest and usually most valuable prospects will convert early in the test.

Once they are taken out, what you are left with is testing the response of the crud, laggards’ late adopters low value customers.

So it’s critical that you figure out who you want to design for: The early hot prospects or the late low value ones. Usually 80% of the revenue will come from the hot prospects.