The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

How an online retailer increased sales by 27% after A/B testing Android phone option

Posted in A/B Split Testing, Case Studies on August 9th, 2011

For eCommerce websites, having the right product mix is very important. If you don’t have products that your customers want, they are going to leave the website and never come back again. And unless you are Amazon or Walmart, you cannot afford to have all sorts of of products to offer (handling inventory, shipping, policies is a logistical challenge so most eCommerce companies specialize in specific niches). So knowing which products to offer is a big challenge for online retailers. This problem becomes even more difficult for the fast-changing technology products such as cell phones.

One of our customers, Mobal, sell international cell phones online. Since the primary product they offer is a cell phone number that works across the globe, choice of handset is a secondary factor in a purchase decision. Or, so they thought! When they A/B tested a new line of handsets by offering additional Android option, they saw sales grow by 27%. This significant increase happened just by adding one new product! No design changes, no new cell phones plans — just a new handset option.

What was A/B tested and why?

Their goal was to determine whether replacing their current deluxe phone with an Android smartphone, or adding an Android smartphone to their current offering would provide an increase in sales over the current 3 phone models they offered. They ran an A/B test on their international cell phones landing page which gets traffic from organic, PPC and direct sources.

Control Page – Budget, Classic and Deluxe Phone

The primary motivation for this A/B test was to see whether there is any demand for smart-phones from international travelers and also to assess the impact of offering a smartphone on their other products. Since international travelers value service and network coverage, they were unsure which variation would convert the best.

Variations tested

They started a test with 3 pages just changing the phone product section of each. One was the original page (control) that you saw above. Other variation looked like below screenshot:

3 options variation (with Android option). 13% increase in sales.

Results: 27% increase of sales

The results were very surprising with the 4 phone option (see a screenshot below) converting 27.77% better than the control page, they felt that an additional phone option may have had an adverse effect on the overall conversion rate but the test proved them wrong. It was also interesting to see that replacing their current deluxe phone with an Android handset saw an increase in conversion over the control page of 13.3%. Clearly, Android phones worked like magic and helped them gain a lot more customers!

4 options variation (with both Android and Deluxe phone option). 27% increase in sales

From this A/B test, they concluded that offering four phone options meant a significant increase in the number of phone sales we had and that there was a demand in the market for smartphones.

Since they used Visual Website Optimizer for their split test, they gave us a testimonial:

Visual Website Optimizer allowed us to simply and efficiently conduct a split URL test for our 3 options. The easy to use dashboard system meant that we could identify which product selection offered us the biggest increase in sales.

So, if you are an eCommerce website, make sure you regularly A/B test your product mix. You never know which new products may increase your overall conversions and sales!

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Steven Kim
August 22, 2011

Thanks for the great example! It’s interesting that the android option replacing deluxe had a 13% lift whereas adding a 4th option created a 27% lift. Did the 27% come entirely from the Android option, or was there a halo effect that created lift for the other 3 phones? Do you think a 5th phone option would be additive?

[...] revenue. If you think you know it all and your e-commerce website has right product mix, read this case study on Mobal, who increased total sales by 27% just by adding a new product to the [...]

[...] How an online retailer increased sales by 27% after A/B testing Android phone option [...]

Leave a comment
Required - not published

nine + = 14

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog

Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to