Visual Website Optimizer

We have published quite a few case studies in past regarding the positive influence of call to action buttons. The testing could be in the form of adding a call to action button when original design didn’t have one, testing a different one or even in the form of  different text for each call to action buttons.

Consolidated Label, one of our customers, A/B tested a new design with a prominent call to action button on their test page. Their original web design did not have any call to action buttons so they tried making an A/B test with one. They did expect an increase in conversions but they didn’t expect it to produce 62% increase in conversions over the original design. The testing helped them gauge the difference a single button can make. Consolidated Label manufactures custom printed labels for retail products in a variety of markets including Food, Beverage, Health & Beauty, and Health markets.

Here are the screenshots of both the Control (original design) and the variation (new design).

Control

Variation (with call to action button) 62% increase in conversions

On asking what lessons they learnt from testing, this is what Tracy Gamlin of Consolidated Label had to say:

Make your desired conversion goal obvious, but harmonious to the rest of the design.  Also, giving people a few places on a page to get to the conversion accommodates different types of users.  For example, the people who just want to get a quote get their button at the top of the page, whereas people who need more substantiation get the opportunity to quote further down the page after they’ve read more copy/seen more images.

Tracy further added, “We’ve found VWO to be immensely valuable for all of our testing needs.  Ultimately, it confirms our feelings with actual data – and many times it even surprises us!”.

If you still haven’t started A/B testing, now is a good time to give Visual Website Optimizer a try with unlimited simultaneous tests.

2 Comments »

We have published numerous case studies regarding user behavior while shopping online. Sometimes, it’s the color of the checkout button that makes the big difference and sometimes it’s A/B testing checkout process that helps our clients make more money. Today, we are publishing a case study that explores effect of having a live chat widget on signups.

A/B testing live chat widget: does it increase signups?

One of our customers, Ez Texting, found that putting a live chat widget helped increase their signups by 31% over the original design (which did not have a live chat widget). Ez Texting provides web-based text messaging services to over 50,000 clients ranging from small organizations to Fortune 500 firms. Here’s how their original signup form looked like:

Control (original design)

Variation (with live chat widget): 31% increase in signups

Ez Texting put an Olark chat widget on the variation and created an A/B test using Visual Website Optimizer. And, as you know, they managed to increase signups by 31%. The chat widget was strategically placed so that if a user has any doubts before making successful signup, he could ask it on the live chat widget.

Lessons from this A/B test

Josh Malin from Ez Texting is a deep believer in power of A/B tests and hence had following to say:

“I know that testing can contradict deeply held beliefs but it is helpful every now and then to go through a test that confirms this. It makes it that much easier in the future to test other long-held beliefs. And of course it’s always great to have solid data to explain the inclusion of some counterintuitive element on our front end sales site to everyone else at the company”

On asking Josh how much Visual Website Optimizer was valuable for their A/B testing, this is what he had to say:

“Absolutely essential. We have used all sorts of manual and automated A/B testing methods, but VWO is the most pleasant to use.”

Hope you liked this case study. To read other success stories, head over to our A/B testing case study library.

4 Comments »

If you sell software online, chances are that your homepage would have one or two prominent call to action buttons. What these buttons say is surprisingly standard across different websites. Typically these buttons have one of the following messages: Learn More, Buy Now, See Plans & Pricing, Try Demo, etc. These buttons are one of the most important elements on homepage and what they say (and what they don’t say) can have material effect on software sales. Hence, it is very important to A/B test these buttons. As a bonus, these buttons are often the easiest elements to test.

A/B testing call to action buttons

One of our customers, Artsy Editor, decided to do A/B test buttons on their homepage. Artsy Editor is a premium WordPress WYSIWYG editor made for bloggers and developers. It provides faster ways to format, add links, upload media and scale images, so you can focus on writing the best content. Hundreds of writers rely on Artsy Editor everyday. Their homepage consists of a call to action sandwiched between a headline and some screenshots of the editor. They decided to A/B test that area and used Visual Website Optimizer for the same.

Variations tested

The goal of their homepage is to route more people to their demo and pricing page. Their hypothesis was that people want to see strong call-to-actions to try out demo and learn about our pricing before actually committing to purchase the product. So, they tested whether a single button can concentrate more clicks or having two buttons with different contrast can give people more options to click on. They also tested different phrases used in the buttons (revealing the price, mentioning the demo, etc.).

They A/B tested with 3 different combinations of buttons. 1) Try Demo & Buy Now for $19.99; 2) Try Demo for Free; 3) Try Demo for Free & Buy Now.

Their variation #1 with a single demo button had a 5% improvement in clickthrough to their demo and 47% improvement in clickthrough to their pricing page. The surprising result here is that even though there wasn’t any Buy Now button, the Try Demo button (indirectly) improved conversions to pricing page by 47%. Here’s how variation #1 looked like:

Variation #1: 47% increase in clickthroughs

Their variation #2 with two buttons (with the emphasis on demo button) had a 7% improvement in clickthrough to demo and 17% improvement in clickthrough to pricing page.

Variation #2: 17% increase in clickthroughs

Variation #3 had two buttons with the price written on the Buy Now button. Surprisingly, this variation had no real improvement in conversions.

Variation #3: no improvement in clickthroughs

Lessons learned from this A/B test

When we asked Artsy Editor about their biggest lesson learned from this A/B test, here is what they said:

One big lesson learned: for software product like us, the primary call-to-action should be the demo/trial, not the checkout/purchase. Because most of the visitors are first-time visitors and they want to focus on learning more about the product before making a purchase decision. If you push too hard at the beginning, they might feel distracted, uncomfortable and leave the site.

Regarding the tool they used, here is what they had to say:

We wouldn’t do anything similar without Visual Website Optimizer. Both creating the test and checking results are done in few clicks.

Artsy Editor say they will continue to test other pages and elements as well, especially pricing and checkout page. If you sell software online, make sure you also A/B test your call to action buttons. It can have dramatic impact on your sales and conversions, just like it did for Artsy Editor.

4 Comments »

One of the questions we often hear from users who are new to A/B testing is what should they test. This does seem like a simple question but it can be quite baffling to the new-comers. This post is one such attempt to narrow down certain options to test, the changes that worked for others might work for you. The change here that we are talking about is testing having a video on homepage or product page. Previous case studies prove that it usually works wonderfully. One such case study is video increases conversion rate of a landing page by 46% and in another one they tested two call to actions ‘Watch the video’ v/s ‘Get Instant Access’ and found out that the video one increased newsletter signups by 28%.

In this latest case study, one of our clients Buy Real Twitter Followers made a variation with a small video explaining about their service on the homepage. This little change helped them increase their sales by 216%. Buy Real Twitter Followers was formed as a response to  the need made by the growing Twitter advertising market. Getting Twitter followers has been a large concern by many network marketers and businesses whose primary aim is to advertise through Twitter.

Visitors usually tend to skip the blocks of text. This case study only goes on to prove that users much rather prefer to see the video for the product validation than reading block of text to get the gist of the product.

Original Page

Variation Page

This is what Rene Harvey had to say about the A/B testing tool they used: Visual Website Optimizer: “VWO was very helpful, it was very easy to use and provides a room for designers such as myself to customize without having to change the original code.”

As always, everything is open to testing and the idea is to find what change benefits the most. Let us know what you think about this case study in the comments below.

8 Comments »

We previously ran an article which showed how long sales pages work. Another previous case study talked about anti-A/B testing on how removing header from our blog increased signups by 60%. Now in this post, I will be talking about how removing content from a landing page and hence shortening it increased sales by 62%. So, what is the real truth? Does more content convert better or less content? The short and sweet answer to this question is that there is no eternal truth. You must A/B test your way to more conversions. Sometimes long copy works, sometimes short copy works.

A/B test case study

AssessmentDay is a leading supplier of practice aptitude tests to graduates and job seekers. They help candidates prepare for their assessment centres and psychometric tests by showing them what to expect. Based on recommendation by their affiliate partner WikiJob (who are also our customer; previously we had featured one case study from WikiJob on how they increased sales by 34%), AssessmentDay ran an A/B split test on their co-branded landing page. Traffic on the landing page was entirely referred traffic from an affiliate’s website. The affiliate’s traffic is all organic and they have banners and text links sending visitors to the landing page.

Cutting corners: variations designed for the landing page

Their original landing page had a screenshot section and a FAQ section. Here’s how it looked:

- Original page -

For variations, they tried out the concept of ‘less is more’. Sometimes people get bored by reading long sales pages and they just navigate away, so we stripped out various parts of the page altogether. Having said that, the original sales page didn’t exactly have lots of text in the first place. For relatively low-ticket price purchases we think people want to quickly see what it is they’re buying and then make a decision; they don’t want to read huge reams of a sales pitch.

A/B test results: 62% increase in sales

The variations we tried were simply removing each of these sections in turn and then removing both sections together, so there were three variations in total. In first variation, they removed the FAQs section. In second variation, they removed the screenshots section.

Split-testing is always surprising! That’s half the fun of it. The result was a 62% improvement by simply removing the FAQ section, or a 56% improvement by removing the screenshot section.

Drop in sales if you remove too much content

What happens if you remove too much content from the landing page? Out of curiosity AssessmentDay then went one step further and deleted both sections of the page but this proved too much and the conversion rate actually dropped by about 3%. Here’s how this variation looked like:

Their hypothesis is that people like to have enough information to help them see what the product is, but any more and they start to get confused or distracted.

Lessons from the A/B test

When we asked if there were any lessons that AssessmentDay learnt from this test, here’s what they said:

• Less is more, up to a point. It’s surprising how you can spend so much time deliberating over the content of a sales page when in fact that section of content actually puts people off and you’re better without it.
• Split testing wins out over gut feel almost every time.

They also had great things to say about the split testing tool they used (Visual Website Optimizer):

Visual Website Optimizer has been extremely valuable, not to mention fun to use. The service has paid for itself several times over.

This was an exciting test with unexpected results. A classic case of trying bold changes and finally getting a much deserved increase in conversions. If you have any comments or feedback on this case study, please let us know!

2 Comments »

Imagine that you are selling cookies online and want to increase sales. What would you do next? One way to increase sales would be to increase traffic that you get but that costs a lot of money. Another way would be to work on optimizing landing pages, product pages and checkout pages. You may test your button colors, call to actions or layout. If you are adventurous, you may move beyond simple changes and start testing entirely different offers or value propositions. Who knows if $5 discount increases sales or probably next-day shipping is more important to convince a customer to gift a box of cookies?

The client: online cookie store

These were exactly the kind of questions that (our certified partner) Trinity Insight wanted to answer for their client SmileyCookie. SmileyCookie is a niche eCommerce store that sells cookies online. They allow you to gift customized cookies to anyone for any occasion (weddings, sports events, corporate events, etc.) What could be a better gift than a box of customized cookies? They worked with Trinity Insight, a consultancy, to help increase their sales and conversions. Trinity Insight used Visual Website Optimizer for A/B testing and then shared their excellent case study with us. (Key point to note is that no internal IT resources of SmileyCookie were used in the test. All of the following was done using Visual Website Optimizer interface).

A/B test challenge

SmileyCookie has a large header location which they use for seasonal or promotional offers. It is at a prominent place and hence catches attention of the visitor. Since an eCommerce store can position itself in various different ways, Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors. They wanted to understand which aspect about SmileyCookie’s service is more important for visitors. Following is a screenshot that shows the header where these value propositions were tested:

Variations tested

Trinity Insight tested five different value propositions and designed their respective creatives. These were five different value propositions:

• Order Today -> Ships Next Business Day
• Want to save $5 OFF your next purchase? SIGN UP NOW ->
• $6.99 Ground Shipping For Your Entire Order
• FREE SHIPPING on any order over $40
• Cookies Made Fresh & Hand Iced For You!

Following screenshot depicts the creatives that were developed:

A/B test results: 41% increase in sales

While the test was running, the overall conversion rate for the site was 8.98% with an average order value was $29.85, yielding a $2.68/per visit value. Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average. Note that this increase was statistically significant at 95% confidence level.

Here’s what Trinity Insight said about the results:

Prior to testing, SmileyCookie could only make educated guesses as to which core value resonated with their customers and an opportunity to generate an additional $1.10/per visitor would have never been realized. By continuing to look at ways to make the site more engaging to their customers and testing these opportunities, SmileyCookie is guaranteed to create an experience that’s better today than it was yesterday as well as learn great lessons along the way about what their customers truly find value in while visiting SmileyCookie.com.

Why next-day shipping worked?

Cookies are perishable items and when you are ordering cookies as a gift or for special occasion, you want them sooner than later. Next day shipping increased sales because it tells the visitor that they don’t have to wait for long to get the cookies. There is no uncertainty and visitors know exactly when they are going to get the cookies. Since these cookies are ordered as a gift or for special occasions, customers may not be very price conscious and hence value proposition regarding discounts or low shipping rate didn’t work that well.

Of course, all this is obvious after we have A/B test results. And in fact, all the reasons proposed above may not be true. Interpreting A/B test results is always hard because if you know why a variation works better, you don’t need to do A/B testing in first place. You can simply implement that variation directly without A/B testing.

But, nevertheless, interpreting A/B test results are useful to see what would work in future or for other similar websites. So, if you sell other such perishable items online (say cakes, food, ice-cream, etc.) emphasizing on how fast your shipping is may increase your sales. Try A/B testing it!

Hope you liked this case study. If you have any comments or feedback, let us know!

4 Comments »

For eCommerce websites, having the right product mix is very important. If you don’t have products that your customers want, they are going to leave the website and never come back again. And unless you are Amazon or Walmart, you cannot afford to have all sorts of of products to offer (handling inventory, shipping, policies is a logistical challenge so most eCommerce companies specialize in specific niches). So knowing which products to offer is a big challenge for online retailers. This problem becomes even more difficult for the fast-changing technology products such as cell phones.

One of our customers, Mobal, sell international cell phones online. Since the primary product they offer is a cell phone number that works across the globe, choice of handset is a secondary factor in a purchase decision. Or, so they thought! When they A/B tested a new line of handsets by offering additional Android option, they saw sales grow by 27%. This significant increase happened just by adding one new product! No design changes, no new cell phones plans — just a new handset option.

What was A/B tested and why?

Their goal was to determine whether replacing their current deluxe phone with an Android smartphone, or adding an Android smartphone to their current offering would provide an increase in sales over the current 3 phone models they offered. They ran an A/B test on their international cell phones landing page which gets traffic from organic, PPC and direct sources.

Control Page – Budget, Classic and Deluxe Phone

The primary motivation for this A/B test was to see whether there is any demand for smart-phones from international travelers and also to assess the impact of offering a smartphone on their other products. Since international travelers value service and network coverage, they were unsure which variation would convert the best.

Variations tested

They started a test with 3 pages just changing the phone product section of each. One was the original page (control) that you saw above. Other variation looked like below screenshot:

3 options variation (with Android option). 13% increase in sales.

Results: 27% increase of sales

The results were very surprising with the 4 phone option (see a screenshot below) converting 27.77% better than the control page, they felt that an additional phone option may have had an adverse effect on the overall conversion rate but the test proved them wrong. It was also interesting to see that replacing their current deluxe phone with an Android handset saw an increase in conversion over the control page of 13.3%. Clearly, Android phones worked like magic and helped them gain a lot more customers!

4 options variation (with both Android and Deluxe phone option). 27% increase in sales

From this A/B test, they concluded that offering four phone options meant a significant increase in the number of phone sales we had and that there was a demand in the market for smartphones.

Since they used Visual Website Optimizer for their split test, they gave us a testimonial:

Visual Website Optimizer allowed us to simply and efficiently conduct a split URL test for our 3 options. The easy to use dashboard system meant that we could identify which product selection offered us the biggest increase in sales.

So, if you are an eCommerce website, make sure you regularly A/B test your product mix. You never know which new products may increase your overall conversions and sales!

3 Comments »

We have a lot of A/B testing case studies on our website now. Most split testing case studies describe a single change and discuss the increase of conversion rate it caused. However, this post is a compilation 3 different A/B tests on a single website. This one is a great case study to see how some companies take multi-pronged approach to successfully optimize different steps of the funnel. These A/B tests were conducted by one of our customers ABNRegistrations.com who provide quality ABN registration (Australian Business Number).

Case Study #1: testing bullet points

The first split test that ABNRegistrations.com did was very simple. On their order page they describe three steps of the ordering process. For the variation, they simply added green ticks next to these steps. Here’s how original order page looked like:

Original order page

Here’s how variation looked like:

Variation (with green ticks)

Result: 2% increase in conversions

As you can see, the change was very trivial. Results were also trivial: a 2% increase in conversions (note that even though increase was small, it was statistically significant). However, this small increase did not deter ABNRegistrations.com. In their own words, they resolved to achieve perfect conversion rate. Here’s what they said:

At a recent staff meeting we set a goal of a 100% conversion rate. We know we won’t achieve this but it provides a focus on ‘continual improvement’. In other words, we don’t set a goal for each test, we just look to improve each time

Case Study #2: testing headline

In the next split test, they tried changing their headline. Originally, they promised a response time of 24 hours. They tested a version which promised a response time of 4 hours. (Of course they weren’t bluffing as they had to change their internal systems to honor this guarantee). Here’s how original headline looked like:

Original headline (with 24 hour response guarantee)

Here’s how variation looked like:

Variation headline (with 4 hour response guarantee)

Result: 10% increase in conversion rate

This (apparently) simple change boosted their conversions by 10%. What’s amazing is that as a side effect of this 4 hour response guarantee, ABNregistrations.com increased their overall quality of service.

Case Study #3: testing order process

On their order page, they tested a better looking order form. Here’s a screenshot:

Improved and better looking order form

Result: 25% increase in conversions

This phenomenal increase occured thanks to complete redesign of signup form. They increased conversion rate from 30.88% to 38.88% (an increase of 25%). You can see this another case study which shows how redesign of sales page increased sales by 20%.

Lessons learned

This case study teaches us that all aspects of conversion funnel can be optimized. From headline to text copy to order form, there is nothing on a website that cannot be tested. So it is important to always keep testing. For ABNRegisterations.com, split testing is ingrained in their culture. The biggest lesson they learned was (in their own words):

Don’t assume anything without testing it. Visual aids/images are just as important as text. Simple changes can make a difference.

When asked how useful was Visual Website Optimizer for their split testing, here is what they had to say:

VWO was incredibly valuable in helping us achieve this, both in terms of the ease of use of the system AND the idea for using the green ticks. I recommend that other business owners and managers take the time to read through VWO’s ideas and brainstorm the things that they might be able to try in a split test. Also, testing should never stop. We are already setting up our next test for the same page.

Needless to say, we are very happy for their success and hope they will continue split testing for many more months to come!

No Comments »

Is pricing information more important than pricing page design? Does design on pricing page influence conversions at all? These are some of the questions that Basekit.com, world’s most flexible website builder, wanted to answer through their latest A/B test. BaseKit allows you to quickly and simply create, host and manage your own website in 100% W3C-compliant HTML. It’s an awesome tool and we are lucky to have them as one of our customers.

Pricing page A/B testing

This case study is about A/B testing of BaseKit’s pricing page and how a redesign of it increased conversions by 25%. They wanted to increase the number of people who visit ‘Buy Now‘ page after visiting their ‘Plans and Pricing‘ page so that was the conversion goal measured in the test. (For their follow up tests, we recommend Basekit to use Visual Website Optimizer’s revenue tracking feature to make sure that not just conversions but total sales is also measured for different variations. This will get a better idea of performance of variation). The traffic that arrived on pricing page (which was tested) is primarily paid, so it is already focused and targeted. Here is how the pricing page originally looked like:

Original pricing page

Redesigned pricing page

In their own words, the variation had “Bolder, brighter, clearer pricing, nicer design, testimonial, more obvious currency selection.” Here’s how the variation looked like:

Redesigned pricing page (25% increase in conversions)

A/B test results and lessons

The new design reached a 95% statistical significance within 24 hours. It was a consistent 25% improvement for the entire duration of the test. They weren’t surprised, but were reassured to see that our improvements had the desired effect. As a follow up test, they are creating yet another redesign of this new pricing page and then do another A/B test. (Continuous A/B testing is key to increasing conversions. Sometimes you get results, sometimes you don’t but if you do lots of A/B tests, you are bound to see some improvements).

Regarding usefulness of Visual Website Optimizer, here’s what Chris Gilfoy, Head of Search at Basekit had to say:

Hugely valuable. I’ve been a fan of VWO for a long time and it was great to take it to a new brand and show how powerful, yet easy to use it is.

This excellent A/B test shows that not just actual pricing details but design of the pricing page is very important. So, when are you going to test your pricing page? (By the way, we are planning to test our pricing page too! Let us know in the comments if you have any ideas for improvement.)

4 Comments »

All ecommerce website owners know how important the add-to-cart button is. They would do anything to have a visitor click on that button because that’s where the actual sales process starts. Because of its importance, many eCommerce retailers start A/B testing variations of that button to improve click through rate. That’s precisely what Trinity Insight, a Visual Website Optimizer customer did for their client: Taylor Gifts.

Trinity Insight is a leading eCommerce consultancy that has helped numerous clients increase conversion rate. (They are also one of our certified agencies). We interviewed Nate Ende of Trinity Insight to talk about the A/B test they recently did to improve add-to-cart clickthrough.

What was the conversion goal of the test?

The premise of this test was to try to improve on the add-to-cart goal of the product page (of TaylorGifts.com).

On which page did you run the test?

The dynamic product page. Here’s an example. (Editor’s note: they used Visual Website Optimizer’s advanced mode to create a test that runs across thousands of dynamic product pages on TaylorGifts.com)

Which part of page did you select for the test and what variations did you test?

Here’s what the original product page looked like:

Original product page

We ran an A/B split test, however we focused mainly on created a buy box with all of the information relevant to the buying decision located in close proximity of the add to cart action. Here’s the variation we tested:

Variation product page (10% increase in clickthroughs)

Why did you think that the variations you created had better chances to beat the original? What were you actually testing in this test?

We felt that presenting this information could help people find the information they needed to make a decision faster and in a more convenient location therefore making them more likely to place the item in their cart.

What results did you get? Were you surprised by the results?

We experienced a 10% lift in the goal conversion on this page and the overall eCommerce conversion rate of the test subjects went from 1.53% on the control to 3.23% on the variation. Needless to say, our client is very happy with the result!

Any lessons which can be derived from your test?

• Placing the standard information we all use to make buying decisions in one easy to scan location makes a lot of sense from a sales standpoint. In traditional retail, ideally, a salesperson would be in close proximity of the item to answer questions about how much it is, and if it’s on sale how much you’re saving. They’d also let you know what other customers thought as so often we use our peers to help us make our decisions.
• Providing valuable eCommerce information near this box may also be a good idea as in how long will it take to get to me and what do I do if I have to return it. More detailed information is great on the page for people looking to make in depth research-based purchases, but the segment of customers who prefer to move quickly through this process will find lots of value in an efficient buy box strategy.

How valuable was Visual Website Optimizer for this test?

VWO was irreplaceable in this test as we’ve done dynamic template tests like this in the past with Google WO and spent an inordinate amount of time creating a custom javaScript and would then need to work with the clients IT team to implement on their site whereas we were able to accomplish the same test with VWO all from within the administration area.

14 Comments »