Visual Website Optimizer

One of the questions we often hear from users who are new to A/B testing is what should they test. This does seem like a simple question but it can be quite baffling to the new-comers. This post is one such attempt to narrow down certain options to test, the changes that worked for others might work for you. The change here that we are talking about is testing having a video on homepage or product page. Previous case studies prove that it usually works wonderfully. One such case study is video increases conversion rate of a landing page by 46% and in another one they tested two call to actions ‘Watch the video’ v/s ‘Get Instant Access’ and found out that the video one increased newsletter signups by 28%.

In this latest case study, one of our clients Buy Real Twitter Followers made a variation with a small video explaining about their service on the homepage. This little change helped them increase their sales by 216%. Buy Real Twitter Followers was formed as a response to  the need made by the growing Twitter advertising market. Getting Twitter followers has been a large concern by many network marketers and businesses whose primary aim is to advertise through Twitter.

Visitors usually tend to skip the blocks of text. This case study only goes on to prove that users much rather prefer to see the video for the product validation than reading block of text to get the gist of the product.

Original Page

Variation Page

This is what Rene Harvey had to say about the A/B testing tool they used: Visual Website Optimizer: “VWO was very helpful, it was very easy to use and provides a room for designers such as myself to customize without having to change the original code.”

As always, everything is open to testing and the idea is to find what change benefits the most. Let us know what you think about this case study in the comments below.

8 Comments »

We previously ran an article which showed how long sales pages work. Another previous case study talked about anti-A/B testing on how removing header from our blog increased signups by 60%. Now in this post, I will be talking about how removing content from a landing page and hence shortening it increased sales by 62%. So, what is the real truth? Does more content convert better or less content? The short and sweet answer to this question is that there is no eternal truth. You must A/B test your way to more conversions. Sometimes long copy works, sometimes short copy works.

A/B test case study

AssessmentDay is a leading supplier of practice aptitude tests to graduates and job seekers. They help candidates prepare for their assessment centres and psychometric tests by showing them what to expect. Based on recommendation by their affiliate partner WikiJob (who are also our customer; previously we had featured one case study from WikiJob on how they increased sales by 34%), AssessmentDay ran an A/B split test on their co-branded landing page. Traffic on the landing page was entirely referred traffic from an affiliate’s website. The affiliate’s traffic is all organic and they have banners and text links sending visitors to the landing page.

Cutting corners: variations designed for the landing page

Their original landing page had a screenshot section and a FAQ section. Here’s how it looked:

- Original page -

For variations, they tried out the concept of ‘less is more’. Sometimes people get bored by reading long sales pages and they just navigate away, so we stripped out various parts of the page altogether. Having said that, the original sales page didn’t exactly have lots of text in the first place. For relatively low-ticket price purchases we think people want to quickly see what it is they’re buying and then make a decision; they don’t want to read huge reams of a sales pitch.

A/B test results: 62% increase in sales

The variations we tried were simply removing each of these sections in turn and then removing both sections together, so there were three variations in total. In first variation, they removed the FAQs section. In second variation, they removed the screenshots section.

Split-testing is always surprising! That’s half the fun of it. The result was a 62% improvement by simply removing the FAQ section, or a 56% improvement by removing the screenshot section.

Drop in sales if you remove too much content

What happens if you remove too much content from the landing page? Out of curiosity AssessmentDay then went one step further and deleted both sections of the page but this proved too much and the conversion rate actually dropped by about 3%. Here’s how this variation looked like:

Their hypothesis is that people like to have enough information to help them see what the product is, but any more and they start to get confused or distracted.

Lessons from the A/B test

When we asked if there were any lessons that AssessmentDay learnt from this test, here’s what they said:

• Less is more, up to a point. It’s surprising how you can spend so much time deliberating over the content of a sales page when in fact that section of content actually puts people off and you’re better without it.
• Split testing wins out over gut feel almost every time.

They also had great things to say about the split testing tool they used (Visual Website Optimizer):

Visual Website Optimizer has been extremely valuable, not to mention fun to use. The service has paid for itself several times over.

This was an exciting test with unexpected results. A classic case of trying bold changes and finally getting a much deserved increase in conversions. If you have any comments or feedback on this case study, please let us know!

2 Comments »

In the past, we have featured a number of A/B testing case studies. In many of those case studies, a number of elements change which contribute to increased sales and conversions. However, this case study is special. Here a single (seemingly trivial) change was responsible for increasing sales by as much as 34%. This is unprecedented and, of course, is an amazing story which demonstrates the power of A/B testing.

That’s why I chatted with Chris Muktar of WikiJob.co.uk, UK’s largest graduate jobs website. Started in 2007, it now attracts over 500,000 visits a month. Started by two friends, it has grown to be a profitable company employing several staff, and proudly boasts many of the world’s largest brands as their clients.

WikiJob use Visual Website Optimizer for A/B testing and below is the interview where Chris talks about how they managed to boost bottomline sales by 34% just by testing a small element.

What conversion goals were measured in the test?

There were two conversion goals. The first was to click through to Paypal checkout. The other was to complete it and actually buy something.

On which page did you run the test?

http://www.wikijob.co.uk/aptitude-tests

What is the traffic source? Organic, direct, PPC, etc.?

Most of the traffic was from our main http://www.WikiJob.co.uk website. The traffic to our website is almost entirely organic.

Which part of page did you select for the test and what variations did you test?

We’ve tested a number of things in the past and had some great results. This time we were testing the effect of three lines of testimonials given by customers and their impact on sales. Below are screenshots.

- Original Page -

- Variation (with screenshots): 34% increase in sales -

Why did you think that the variations you created had better chances to beat the original? What were you actually testing in this test?

A previous test displayed the same testimonials further down the page and had no discernable impact on conversion. However, it’s commonly thought that social proof can help in the sales process, so we wanted to leave nothing to chance and test it.

What results did you get? Were you surprised by the results?

Testimonials increased sales by 34%. The testimonials we used are very ‘sober’ (compared with the overly enthusiastic ones you so often see in marketing literature). The test results were surprising. Although such increases of sales can be quite normal in split testing, I did not think that testimonials would make such a difference (and indeed put off testing them, thinking they were irrelevant). The increase in revenue was very substantial.

Any lessons which can be derived from your test?

Social proof is important, and it is important to test everything. I may try putting some small pictures next to each testimonial to see if this improves things. [Editor's note: do that! Our previous split tests show that human pictures can potentially double conversion rates]

How valuable was Visual Website Optimizer for this test?

Visual Website Optimizer has been invaluable. You pay for the service once, but the increase profits and revenues continue into perpetuity. Through the use of several iterations of tests, we have increased sales threefold.

16 Comments »

ReplaceDirect, a Visual Website Optimizer customer, is a successful Dutch e-commerce site. It’s one of the biggest retailers in the Netherlands in the field of parts and accessories for notebooks and other mobile equipment. The main focus is on energy products like batteries and power supplies, but the product range is quite extensive.

They recently did an A/B test using Visual Website Optimizer where the winning variation reduced cart abandonment by 25%. For e-commerce sites, cart abandonment is one of the most frustrating aspects of business. After all, there’s no company that likes a potential customer to abandon the purchase after spending tons of time on site research and selecting the right products. Fortunately, cart abandonment can be fixed using A/B testing and this case study demonstrates how ReplaceDirect did it.

A recent study by Forrester found out that shipping costs rank as the number one reason for shopping cart abandonment. Combined with other reasons found out in the research, it can be concluded that visitors abandon their shopping cart because they are unsure about additional costs or surprised to find out (only after entering the payment process) that they are actually charged more than they expected. In order to prevent such surprises, ReplaceDirect already tells the customer in an early stage that no shipping costs will be added. So the customer is clearly informed right from the start. This is exactly what ReplaceDirect wanted to test: will it help reduce cart abandonment when a clear order overview is provided during the initial steps of checkout?

Another cart abandonment study by Paypal

They tested their second step of the checkout procedure. It’s the page where customers are asked to fill out their personal information and shipping address. ReplaceDirect made several changes to the page. Probably the most important change was the insertion of an order overview, comprising the product(s), total costs and delivery date. ReplaceDirect also mentions several benefits at this page; these were changed to more relevant benefits specifically matching this step in the order process. Finally the layouts of the page and the form were changed to yield a cleaner look. Several fields, which were not absolutely necessary, were removed. Following are the screenshots:

Control page

Variation page: 25% reduction in cart abandonment

Their new version performed significantly better, reducing cart abandonments by 25%. As a direct result of this reduction, they saw a 12% increase in sales (14% by the time of pushing all new traffic to the new page). ReplaceDirect says they expected the new version to perform better, but an increase that high turned out to be a pleasant surprise. Well, it is always nice to have a 14% increase in sales without releasing any new products or without spending money on ways to get more traffic to the website.

When we asked them if they had any lessons for other e-commerce sites; here is what they had to say:

Try to give the customers exactly the information they need at that particular page or section and leave out the redundant. Always put yourself in the customer’s shoes. This may not be easy so: test it!

ReplaceDirect also had some praise for Visual Website Optimizer:

VWO was very valuable. The ease to set up a test and real-time results made it great to work with.

We are very happy that ReplaceDirect reported these great results by doing an A/B test using our tool.
If you manage an e-commerce site without doing any A/B split testing, you are certainly leaving a lot of money on the table.

No Comments »