How to calculate number of combinations?
The total number of combinations in a test can be easily found out by muliplying the number of variations (including control) you have in each section. For example, if you have two sections in your website which you want to vary: headline and ad copy. For headline, you have 3 variations (control, big headline, headline in red color). For ad copy, you have 2 variations (control, short ad copy). The total number of combinations possible using these variations is: 2 x 3 = 6. Similarly, if we had 3 sections with variations 3, 4, 2; the total number of combinations become 3 x 4 x 2 = 24. Also, obviously, if we have a single section in the test, number of combinations is equal to number of variations.
How long should you run the test?
The calculator below allows you to calculate maximum duration (in days) for which the test should be run. If within this duration no winning combination is found, it can be easily concluded that there is no combination which beats the control (and hence test can be stopped). Even though exact test duration depends on a lot of external factors, entering basic information about your test will give you a rough idea of the duration.
