The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

Categories

Find Articles

I created a test in which control and variation are exactly the same, why am I getting difference in results?

Actually, unless difference is statistically significant, you shouldn’t bother about the results. Due to randomness in A/B testing, percentage difference in conversion rate can occur. For example, a variation may have 10% conversion rate and control 8% conversion rate just by chance. As you test more traffic, this difference should come down and ideally with infinite number of traffic, conversion rate should be exactly same.

Following are some of the important points to consider when you get different results for exactly same control and variation.

  1. In A/A test dont just look for percentage improvement, but statistical significance too, if no statistical significance, don’t read too much in results.
  2. Even if statistical significance arrives, please allow the test to run for at least 7 more days.
  3. In case you are still doubtful of the results, you can set thresholds for chance to beat original to 99%.
VN:F [1.9.22_1171]
Rating: 2.0/5 (3 votes cast)
I created a test in which control and variation are exactly the same, why am I getting difference in results? , 2.0 out of 5 based on 3 ratings
No Comments

No comments yet.

Leave a comment →


Raise Support Ticket

Top

Raise a Support Ticket!

Your Name

Your Email

Your Phone

Your Query

Or, contact us through email support@wingify.com

A B Testing Tool