The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

How WriteWork.com increased sales by 50% & doubled conversions by A/B testing a radical new design

Posted in A/B Split Testing, Case Studies on February 22nd, 2011

Sometimes it takes a radical redesign to get big results. ClickLab is a Brazilian agency specialized in Conversion Rate Optimization and they used Visual Website Optimizer to test a radical redesign of the primary page on WriteWork.com, a popular essay website for students.

They chose to focus on the page which receives most entrance traffic. The traffic is nearly exclusively organic and the page has been struggling with high bounce rates for a while. The objective was to increase engagement and get more people further down the funnel.

The Arsenal

After crunching user surveys, getting feedback from tools such as UserTesting.com, Fivesecondtest.com, Gazehawk.com and ConceptFeedback.com a radical redesign was developed and recently tested using Visual Website Optimizer.

Targeting A/B test to a specific visitor segment

One challenge was how to go about testing the new design. Pretty much everything was different, from header to footer. This means a user might land on the homepage, which has one look, then click on to the redesigned page, which has a different look.

Luckily this is easily solved in Visual Website Optimizer. Since the primary objective was to increase engagement on the landing page, the following segmentation trick was used:

By restricting the test to only the visitors landing on the page, engagement could be measured without having to worry about the design not being consistent throughout the site.

Design Variations

The original page didn’t communicate what the benefits of the service are. Following is how original design looked like:

After reviewing over 1,000 survey responses it became clear what the real benefits were and then these benefits were communicated much more directly in the variation. Secondly, social proof and various credibility indicators were introduced. Here is how variation looked like:

A/B test Results

The result of the test was that engagement more than doubled. One important thing to note is that engagement in Visual Website Optimizer (VWO) only measures clicks and not submissions of forms, for example using a search form (for this reason searches were measured separately).

The great thing about VWO is that you can measure multiple goals. This meant it’s possible to see exactly where the engagement was happening – the biggest increase was indeed clicks on the CTA. The four top buttons accounted for less 1% in the engagement increase.

The secondary objective was to get people to click on the primary CTA and here results increased by 144%.

This meant that more than twice as many people continued on to the payment page. More payment page views doesn’t automatically more payment, but a follow up test showed that payments went up by over 50%.

Actually, although the results were fantastic, they weren’t that surprised. After going over the surveys from users it became so clear that the original really did a terrible job at selling the service.

Lessons from the A/B test

What allowed them these fantastic results was that they started with the user in mind; doing everything they could to understand him/her. Understand their wants, their worries and their needs. They also looked very closely at the language used on the website. For example, WriteWork has always used the expression “overcome writer’s block”, but no users used these words. Instead users wanted to hear “get started” and “get inspiration”. So now they use that language on the website.

Role of Visual Website Optimizer

Jens Schriver from ClickLab gave Visual Website Optimizer a nice, short testimonial:

It was a breeze to setup this A/B test and segment it. We’ve used Google Website Optimizer many times in the past, but – if we can avoid it – we are not going back :)

Hope you liked this case study. Our online case study library has many more A/B split testing case studies. And if you’d like to do a similar A/B test on your website, we have a free 30 day (no obligation) trial of Visual Website Optimizer.

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

11 Comments
User
February 22, 2011

“By restricting the test to only the visitors landing on the page, engagement could be measured without having to worry about the design not being consistent throughout the site.”

But what happens when the user clicks to the next page? Then he will see the old design, right?

Would it be possible to change ALL pages of a site in a big AB-test (without different urls and by using the visual interface)?

Jens Schriver
February 22, 2011

“But what happens when the user clicks to the next page? Then he will see the old design, right?”

That’s correct, but as soon as he clicks we have measured both engagement and CTA clicks.

“Would it be possible to change ALL pages of a site in a big AB-test (without different urls and by using the visual interface)?”

This can be solved by letting all links on the variation page point to a different sub-domain which has the new design. It requires some work, but it’s possible.

Alamgir Kahn
February 23, 2011

What did the survey ask that lead to the original insight and the decision that some kind of a change was needed?

Awesome case study!

I’d definitely say that’s a radical redesign if I ever saw one, and much, much better.

(As the results show)

I especially like how CLEAR the new design is. It’s much easier to understand what the site is about at the blink of an eye.

Jeremy Reeves

Jens Schriver
February 23, 2011

“What did the survey ask that lead to the original insight and the decision that some kind of a change was needed?”

When going over the surveys we were looking for answers to these questions:
1. For those that paid: why did they pay?
2. For those that didn’t pay: why didn’t they pay?
3. What language are the users actually using?
4. For those that had been members for a while, what was the main benefit they’d gotten out of the survey?

These answers can be extracted using simple email surveys -send an email to those who just paid, or those that registered, but didn’t pay. Offer something free, or the chance to win something, this will greatly increase chance of response. We also used KISSinsights which gave some good feedback.

It was particularly the answers to question 4 that showed us something needed to change in the communication. People were simply using the site in ways that nobody had imagined. Very interesting!

@jens_schriver
ClickLab

[...] Daha önce yapılmış örnek çalışmalara göz atabilirsiniz.A/B Testinde Başarılı Örnekler:WriteWork.comOfficial Vancouver 2010 Olympic Store NutshellMath Homework Help JML DirectThe Split Testing [...]

[...] WriteWork.com [...]

price action trading
March 14, 2011

Incredible article on split testing. I wonder if there is a way to test price. The action of the user clicking to accept a price or abandoning a purchase would be an interesting event to track.

Jens Schriver
March 14, 2011

Re: previous comment:

We’ve tested prices many times over the years. That’s easy (especially with a tool like VWO). The real challenge is to find what price points to test.

A great service is http://price.intelligent.ly/ – we’ve used this service to find optimum prices and recommend it.

@jens_schriver
ClickLab

[...] How WriteWork.com increased sales by 50% & doubled conversions by A/B testing a radical new desi… [...]

[...] WriteWork.com doubled their conversions and increase sales by 50% by A/B testing a new design.(source) [...]

Leave a comment
Required
Required - not published


6 − = five

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com