- ►Introduction (11)
- ►Features (44)
- ►Common Functions (17)
- ►Use Cases (8)
- ►Common Problems (24)
- ►Technical Integration (9)
- ►Plugins (12)
- ►Accounts and Billing (10)
- ►FAQs (46)
- ►Uncategorized (1)
How we decide to split?
Actually whenever a visitor comes to your website, we flip a coin (that is, generate a random number). In case the result is Heads, we show version A, otherwise we show version B. In other words, we choose randomly which version to show to the visitor.
Due to randomness, initially split may not be exact 50/50 (or variations / control may be getting unequal number of visitors). However, in long run, the traffic split should get closer to 50/50.
Special note for split URL test: in split URL, if variation is chosen, a redirect happens and variation page is shown to the user. We have observed that in 5-10% cases, this redirect fails to happen, so you may notice that variations may be receiving slightly less number of visitors as compared to control. This slight difference is due to faliure to redirect which may be due to some plugins installed by visitor, or because of browser settings.
Your test is valid even if split is not equal
Testing works on proportions. So whenever we compare two variations (or a variation and control), we always compare their conversion rate, and not number of visitors they got. So, even if your split is not exact 50/50, you can rest assured that your test is still valid and you can still take decisions based on the results.
Split ratio has very little to do with statistical significance, so you can comfortably ignore minor differences in split ratio.