The Next Generation of Visual Website Optimizer is launching on 29th April 2014 See What's Coming

How changing a single word increased click through rate by 161%

Posted in Case Studies on October 4th, 2012

Veeam Software, an Elite VMware Technology Alliance partner and a Microsoft managed partner, develops products for virtual infrastructure management and data protection. Veeam is a multinational company with global headquarters in Baar, Switzerland that has more than 800 employees and 39,000 customers worldwide. They use Visual Website Optimizer to run optimization tests on their website http://veeam.com. This case study is about one of their A/B testing wins.

Background

In December 2011, they used KissInsights (now Qualaroo) to run a survey targeted at all visitors to their product pages. The question asked was “What other information would you like to see on this page?” and a lot of answers said “Pricing”. The company does not publish pricing information on their pages because they sell through partners and discounts given out by different partners may vary. However, they did have a “Request a quote” link that led to a Sales Inquiry form. The goal of the test was to increase click through rate to the sales inquiry page.

Hypothesis

Changing the link text from “Request a quote” to “Request pricing” will increase the click through rate.

The test

Based on the survey results, they used Visual Website Optimizer to A/B  test between “Request a quote” and “Request pricing“.

Control – “Request a quote”

Variation – “Request pricing” 

 The result

A straight 161.66% increase in click through rate from 0.54% to 1.40% with 100% statistical confidence (VWO reports 99.9x% as 100%). That’s a huge increase, and it must have taken only about 2 minutes to setup the test. Some of you might feel that this result would be more concrete if the number of leads generated would drastically increase. Well, you’re right. However, optimizing a website involves tweaking multiple steps of the sales funnel. The next step after optimizing the click through rate to a sales inquiry page would be to try and increase the form submit rate.

Insights derived

Veeam have done a lot right here. The first was starting off by gathering data and feedback. The best conversion (and scientific) experiments start with observation and asking for feedback. Then, they listened to their customers. As you probably know, it’s difficult to go wrong if you listen closely to your customers and change accordingly. The final bit of actionable insight is this: be direct and clear in your communication. This is specially true of the web, where consumers are attuned to filtering out anything except the core information they’re looking for. If they don’t quickly find it, they simply bounce.

Your views?

What do you think of the test? What would you do in the next steps? Also, here is the Sales Inquiry page. What suggestions would you give Olga, the Marketing Manager to optimize the form to increase submit rates? Please share your thoughts and ideas in the comments.

You can participate in the HackerNews discussion here.

Siddharth Deswal

I do marketing at Visual Website Optimizer.

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

6 Comments
beatgammit
October 6, 2012

Technically that’s two words, and “quote” and “pricing” have different implications. Quote implies that price is dependant on the application, whereas pricing implies a set price that the requester can choose between.

I think this is the real insight. If your prices are fixed, then pricing is a better option. If your prices are dependent on the application (needs review by a human), quote is more informative.

Whenever I see “quote”, I assume that it’s more expensive than I can afford, as it’s probably a highly specialized product. When I see pricing, I assume that they just don’t want to reveal prices on the page because of competition.

Unrelated:
This comment box is terrible. My comment didn’t fit in the box, and there was no scroll bar (text just disappeared). Dragging the window bigger caused weird problems (I’m currently typing in a gray area below the white box). This is not acceptable for a site named “visualwebsiteoptimizer”…

Siddharth Deswal
October 6, 2012

Beatgammit,

If you’ll take some time to submit the form (we did…for purposes of this case study), a sales person from Veeam will get back to discuss your requirements.

So “quote” is probably the more accurate word here, but not what visitors are looking for. Therefore, it’s probably slightly misleading but better for the conversion rates.

About the comment box: you’re right, it is without accepted UI elements that have been around since the 1980s. Will sort this. Thanks for pointing it out.

Jeremy Reeves
October 8, 2012

Great test! I’d love to know how long they let the test run/how many visitors. I have a feeling it wasn’t as many as it should (i.e. it may have still been a winner but not such a big winner), but I don’t like to assume ;)

Either way, great reminder to test the call to action!

Jeremy Reeves
http://www.101conversiontips.com/free-video/

Siddharth Deswal
October 8, 2012

Jeremy, there were about 3500 visitors on both Control and Variation.

Graham Nichols
November 6, 2012

Siddharth, I noticed when clicking Jeremy’s website link that it opens in the same window. Thus taking me away from the VWO nest of goodness. You may want to change this, as giving visitors a site exit defies what VWO stands for.

Who would want to leave all this great information behind anyway?

Thanks for another interesting, and informative, case study.

Keep spreading the love.

Siddharth Deswal
November 6, 2012

Graham, thanks for your comment. We had a major discussion in office about links opening in other windows (based on this article http://uxdesign.smashingmagazine.com/2008/07/01/should-links-open-in-new-windows/), but I guess you’ve settled it with your feedback :)

Also, thanks so much for the kind words! Really happy to hear the the case studies and articles are useful for you.

Leave a comment
Required
Required - not published


five − 2 =

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com