Sometimes it takes a radical redesign to get big results. ClickLab is a Brazilian agency specialized in Conversion Rate Optimization and they used Visual Website Optimizer to test a radical redesign of the primary page on WriteWork.com, a popular essay website for students.
They chose to focus on the page which receives most entrance traffic. The traffic is nearly exclusively organic and the page has been struggling with high bounce rates for a while. The objective was to increase engagement and get more people further down the funnel.
After crunching user surveys, getting feedback from tools such as UserTesting.com, Fivesecondtest.com, Gazehawk.com and ConceptFeedback.com a radical redesign was developed and recently tested using Visual Website Optimizer.
One challenge was how to go about testing the new design. Pretty much everything was different, from header to footer. This means a user might land on the homepage, which has one look, then click on to the redesigned page, which has a different look.
Luckily this is easily solved in Visual Website Optimizer. Since the primary objective was to increase engagement on the landing page, the following segmentation trick was used:
By restricting the test to only the visitors landing on the page, engagement could be measured without having to worry about the design not being consistent throughout the site.
The original page didn’t communicate what the benefits of the service are. Following is how original design looked like:
After reviewing over 1,000 survey responses it became clear what the real benefits were and then these benefits were communicated much more directly in the variation. Secondly, social proof and various credibility indicators were introduced. Here is how variation looked like:
The result of the test was that engagement more than doubled. One important thing to note is that engagement in Visual Website Optimizer (VWO) only measures clicks and not submissions of forms, for example using a search form (for this reason searches were measured separately).
The great thing about VWO is that you can measure multiple goals. This meant it’s possible to see exactly where the engagement was happening – the biggest increase was indeed clicks on the CTA. The four top buttons accounted for less 1% in the engagement increase.
The secondary objective was to get people to click on the primary CTA and here results increased by 144%.
This meant that more than twice as many people continued on to the payment page. More payment page views doesn’t automatically more payment, but a follow up test showed that payments went up by over 50%.
Actually, although the results were fantastic, they weren’t that surprised. After going over the surveys from users it became so clear that the original really did a terrible job at selling the service.
What allowed them these fantastic results was that they started with the user in mind; doing everything they could to understand him/her. Understand their wants, their worries and their needs. They also looked very closely at the language used on the website. For example, WriteWork has always used the expression “overcome writer’s block”, but no users used these words. Instead users wanted to hear “get started” and “get inspiration”. So now they use that language on the website.
Jens Schriver from ClickLab gave Visual Website Optimizer a nice, short testimonial:
It was a breeze to setup this A/B test and segment it. We’ve used Google Website Optimizer many times in the past, but – if we can avoid it – we are not going back
Hope you liked this case study. Our online case study library has many more A/B split testing case studies. And if you’d like to do a similar A/B test on your website, we have a free 30 day (no obligation) trial of Visual Website Optimizer.
RSS feed for comments on this post. TrackBack URL
“By restricting the test to only the visitors landing on the page, engagement could be measured without having to worry about the design not being consistent throughout the site.”
But what happens when the user clicks to the next page? Then he will see the old design, right?
Would it be possible to change ALL pages of a site in a big AB-test (without different urls and by using the visual interface)?
Comment by User — February 22, 2011 @ 10:47 pm
“But what happens when the user clicks to the next page? Then he will see the old design, right?”
That’s correct, but as soon as he clicks we have measured both engagement and CTA clicks.
“Would it be possible to change ALL pages of a site in a big AB-test (without different urls and by using the visual interface)?”
This can be solved by letting all links on the variation page point to a different sub-domain which has the new design. It requires some work, but it’s possible.
Comment by Jens Schriver — February 22, 2011 @ 11:43 pm
What did the survey ask that lead to the original insight and the decision that some kind of a change was needed?
Comment by Alamgir Kahn — February 23, 2011 @ 1:04 pm
Awesome case study!
I’d definitely say that’s a radical redesign if I ever saw one, and much, much better.
(As the results show)
I especially like how CLEAR the new design is. It’s much easier to understand what the site is about at the blink of an eye.
Jeremy Reeves
Comment by Freelance copywriter Jeremy Reeves — February 23, 2011 @ 6:25 pm
“What did the survey ask that lead to the original insight and the decision that some kind of a change was needed?”
When going over the surveys we were looking for answers to these questions:
1. For those that paid: why did they pay?
2. For those that didn’t pay: why didn’t they pay?
3. What language are the users actually using?
4. For those that had been members for a while, what was the main benefit they’d gotten out of the survey?
These answers can be extracted using simple email surveys -send an email to those who just paid, or those that registered, but didn’t pay. Offer something free, or the chance to win something, this will greatly increase chance of response. We also used KISSinsights which gave some good feedback.
It was particularly the answers to question 4 that showed us something needed to change in the communication. People were simply using the site in ways that nobody had imagined. Very interesting!
@jens_schriver
ClickLab
Comment by Jens Schriver — February 23, 2011 @ 6:42 pm
[...] Daha önce yapılmış örnek çalışmalara göz atabilirsiniz.A/B Testinde Başarılı Örnekler:WriteWork.comOfficial Vancouver 2010 Olympic Store NutshellMath Homework Help JML DirectThe Split Testing [...]
Pingback by A/B Testi Nedir? A/B Test Araçları, Örnekler ve Kaynaklar — February 25, 2011 @ 6:37 pm
[...] WriteWork.com [...]
Pingback by #A/B testing case studies @econsultancy « Burcin Kaya Blog — February 26, 2011 @ 3:01 am
Incredible article on split testing. I wonder if there is a way to test price. The action of the user clicking to accept a price or abandoning a purchase would be an interesting event to track.
Comment by price action trading — March 14, 2011 @ 11:01 pm
Re: previous comment:
We’ve tested prices many times over the years. That’s easy (especially with a tool like VWO). The real challenge is to find what price points to test.
A great service is http://price.intelligent.ly/ – we’ve used this service to find optimum prices and recommend it.
@jens_schriver
ClickLab
Comment by Jens Schriver — March 14, 2011 @ 11:35 pm
[...] How WriteWork.com increased sales by 50% & doubled conversions by A/B testing a radical new desi… [...]
Pingback by Selling cookies online? Next | MegaSoul — August 24, 2011 @ 4:50 pm