UPDATE: The contest is over and here are the results.
Let’s play a little game. If I say that one of the Visual Website Optimizer customers increased opt-in signup rate by 28% simply by changing a single button, can you guess which one was it? To make things simple, I will even tell you it was one of these two: ‘Get Instant Access’ or ‘Watch a Video’.
Challenge: Guess which version was the actual winner in this A/B test.
Prize: Out of all the correct entries, one lucky person will win a full year of $249/month Visual Website Optimizer subscription!! Twelve months of our Large Agency plan is worth $3000 (our customer base includes some of the most well known enterprises in the world and hundreds of startups & small businesses).
How to enter?: Simply leave a comment on blog or tweet or post on Facebook about which variation you think increased signups by 28%. If you tweet, either use the tweet boxes below or make sure you cc @wingify so that we can track your entry.
When will be the winner chosen?: On January 6th 2011, along with the actual case study, we will announce the winner of this contest.
About this A/B test case study: the objective of the landing page was to convert visitors to email newsletter subscribers. The email newsletter is about dating tips and tricks. Upon signup, visitors also get access to a free video on dating. One version of the button promised them instant access to the video. Other version simply asked them to Watch the video. (See the screenshots below) You have to guess which one increased conversions by 28%.
Version A: Get Instant Access
Version B: Watch the Video
Remember, the prize for this contest is a full year of $249/mo Visual Website Optimizer subscription worth $3000! So, enter take part into contest now either by tweeting your choice above or leaving a comment below or posting it on Facebook. (Bonus: if you do all three, your entry will be counted thrice.)
Good luck! Hope you have a happy new year.
Yay! It is holiday time again. We all have some exciting, fun-filled weeks ahead. Santa Claus, Snow, Lights, Trees, Decorations, oh we absolutely love this time of the year. The whole team at Visual Website Optimizer wishes you Merry Christmas and Happy New Year 2011. May you have best of the times with your friends and family.
We wanted to celebrate this occasion with a teeny-tiny gift for you:
For availing this gift, all you need to do is to login to your Visual Website Optimizer account before 7th January 2011. That’s it. No gimmicks, no fluff. Just a honest gift from our side 
We hope you enjoy the holidays and have a prosperous 2011.
Boy, what a year 2010 has been for A/B testing industry and Visual Website Optimizer. We saw tens and hundreds of small/medium sized businesses increase sales and conversions on their websites. Our repository of A/B testing case studies is the proof that A/B testing works! To celebrate the excitement of 2011, I have complied the list of our best A/B testing case studies in 2010 (in order of bestness). Hope they will inspire you to kick-start A/B testing campaign in your organization in 2011.
Original Landing Page (with embedded signup form)
Variation (no signup form): 60% increase in signups!
Who would have thought that removing a signup form from signup page can actually increase conversions by 60%? Vendio, an Alibaba.com company, used Visual Website Optimizer to do an A/B test on their signup landing page. In one version, they removed the signup form altogether and instead wrote about benefits of signing up. The end result was that the version with no signup form eventually saw 60% more signups. Click here to read the full case study (includes screenshots)
Control: 14.5% conversion rate
Variation: 28% improvement conversion rate!
Soocial tested multiple variations of their sign up button in order to increase clickthroughs to the registration form. Variations included call-to-actions such as “Sign up for free”, “Free sign up”, etc. The A/B test results revealed that one variation increased conversions dramatically. Click here to read the full case study (includes screenshots)
Control (paintings)
Variation (photos): 95% increase in CTR!
In two separate A/B tests, it was found out that human face photos doubled the conversion rate on a website. The result was striking because the tests were conducted by two completely unrelated companies, yet they arrived on the same conclusion. Click here to read the full case study (includes screenshots)
Original (control)
Redesigned (variation): 85% increase in downloads!
You Need A Budget makes easy-to-use personal financial management software. They split tested their product tour page by including more screenshots, adding a prominent testimonial and reducing text clutter. The redesigned page increased downloads of the free trial by 85%. Click here to read the full case study (includes screenshots)
Original version
Redesigned version: 20% increase in sales!
AquaSoft redesigned their sales page by giving it a modern, clean look. They also added trust building elements such as money-back guarantee, assurance seal, etc. Using Visual Website Optimizer, they effortlessly tested old v/s new sales page. After multiple phases of testing, they successfully increased total sales by 20%. Click here to read the full case study (includes screenshots)
Well, we already have a lineup of many interesting split testing case studies. From product pages to eCommerce sites. From effect of video on signups to effect of number of steps in checkout process on sales. We have a case study lined up for all these situations.
To keep updated with our latest A/B testing case studies, subscribe blog updates through email (powered by Feedburner).
Let’s admit it: your boss has certain opinions on the website design, the usability consultant has some “best-practices” suggestions and the sales team wants specific changes to increase leads. Heck, even your office receptionist thinks that you could add Facebook and Twitter buttons on the homepage. Needless to say, everyone in your office has an opinion on website design and if you are the one with Photoshop/HTML skills, it can drive you crazy fairly quickly. So, how to cope up with suggestion/feedback overload all the while trying to increase conversion rate on the website?
Rules of the game are pretty simple. You declare a split testing tournament at the Friday office lunch to win a booty of say $100 (or any other motivating prize). Anyone in your office (or team) with an opinion on site design gets to create a variation for a split test. You then pit all variations against each other in a live split test and quite obviously the person who came up with winning variation wins the prize booty!
This strategy brings out the best talent from across the team and helps you test diverse ideas at once. Since inputs for improving conversion rate and sales comes from different team members, this also helps you avoid designer’s block (of coming up with highly converting ideas and designs).
Visual Website Optimizer is the perfect tool for playing this game. Here’s why:
Do you think this is a killer idea? Have you ever tried this strategy in your office? At Wingify, we keep playing this game (after all, we make split testing tool). You also try this and let us know your thoughts and opinions. We have heard that the split testing game works like wonder to increase conversion rate and sales.
One of the Visual Website Optimizer users, Jason Thompson did a great job in integrating VWO with Omniture SiteCatalyst. (We already have a Google Analytics plugin).
We recently updated the plugin for Site Catalyst and decided to write the instructions on our own blog (though the original credit for the code goes to Jason).
You need to designate one of your Custom Conversion variables as your optimization tracking variable and copy-paste following code in doPlugins section of your s_code.js file. (Also make sure VWO code snippet is located before s_code.js on your page)
try { if(typeof(_vis_opt_settings_loaded) == "boolean"){ var _combination = _vis_opt_readCookie('_vis_opt_exp_'+_vis_opt_experiment_id+'_combi'); if(typeof(_vis_opt_comb_name[_combination]) != "undefined"){ s.eVar55= _vis_opt_experiment_id + ':' + _vis_opt_comb_name[_combination]; } } } catch(err) { }
That’s all is what is required to start importing Visual Website Optimizer variation data into your Omniture SiteCatalyst.
Following is a sample report (fictitious data):
That’s it. I hope you get a wealth of additional data from our SiteCatalyst plugin.
Many people seem to get confused about the terms A/B testing, split testing and multivariate testing. Essentially, A/B testing and split testing are the same concepts but multivariate testing is different. So for all practical concepts, consider A/B Testing = Split Testing. The following table will illustrate difference between A/B testing (or split testing) and Multivariate Testing:
| A/B Testing | Multivariate Testing | |
| What is it? |
In A/B testing you split traffic amongst two or more completely different versions of a webpage (landing page, home page, etc.) The variations of your original page can differ in any manner. You can either just change the headline; or you can even change entire design, layout, offer and what not in the variations. What you change in A/B test is only limited by your creativity. For example, if you are A/B testing on your landing page you may want to create one version with 15% discount, the other with free shipping and one with the same offer but different page design. |
In multivariate test, you identify a few key areas/sections of a page and then create variations for those sections specifically (as opposed to creating variations of whole page in an A/B split test). So for example, in multivariate test you can choose to create different variations for 2 different sections: headline and image. A multivariate testing software will combine all these section specific variations to generate unique versions of page to be tested and then simply split traffic amongst those versions. That is, you will get these variations of the page: Headline1_Image1, Headline1_Image2, Headline2_Image1, Headline2_Image2. |
| How it works? |  |
 |
| Traffic Required |
Relatively Less |
Huge |
| Best used for |
Testing radically different ideas for conversion rate optimization |
Optimizing and refining an existing landing page or homepage without doing significant investment in redesign |
| Example Case Study |
Signups increased by 60% after actually removing the signup form |
I hope that was a good comparison between A/B testing and multivariate testing. If you think I left any point, please let me know. I will update the comparison table accordingly.
Last year, at the dawn of 2010 I wrote four reasons why 2010 is going to be a year of A/B split testing. Now that 2010 is coming to a close, I thought it will be great to revisit what happened in our tiny little (yet growing) industry of split testing and conversion rate optimization. Undoubtedly, the year saw explosive growth in the industry (and also for our A/B testing tool – Visual Website Optimizer). It was incredibly hard to pick 10 best A/B testing resources (not because there weren’t any, but rather there were simply too many great tools, websites, forums, tutorials, etc.). Anyhow, I managed to compile following is the list of top 10 resources (in no particular order):
WhichTestWon.com is an interesting website which features a new A/B split test every week. You can vote and discuss about which variation won in the test. It is a fun way to keep learning what usually works for increasing conversion rate on website.
ABTests.com is an open forum where anyone can share and discuss their A/B test results. ABTests.com and WhichTestWon.com are two great websites when it comes to reading real-life A/B testing case studies. You simply can’t get enough of them!
The Ultimate Guide To A/B Testing is a guest post I wrote for Smashing Magazine. I had tried to make this post as the definite resource for anyone who wants to get started with A/B testing. Judging from #2 position on Google for ‘A/B testing’ (#1 is Wikipedia), I think I have achieved my goal with that guest post.
#crochat is a hashtag used by a group of conversion rate optimization enthusiasts to chat on this topic every Thursday. I have seen everyone from experts to vendors to beginners participating in the chat and discussing interesting topics related to A/B testing.
MarketingExperiments.com is an organization that does numerous split and multivariate tests every week and have been doing so for years now. They have built an impressive research library which details how they improved website conversions and sales on different kinds of websites: eCommerce, publishers, lead-gen landing pages, etc.
Conversion Conference is the first and only conference dedicated to the art of conversion rate optimization. The next edition will be held in San Francisco in March, 2011.
WhichMVT.com is a site where you can compare different multivariate testing tools (including Visual Website Optimizer). You can also read or leave review of different tools. (Just in case you happen to use VWO and love it, please take 5 minutes out to leave a review here).
Online Behavior is a new magazine by industry experts which features columns on behavioral targeting, web analytics and testing & optimization. All articles are of top-notch quality. If you are interested in conversion rate optimization, subscribing to their feed is a must.
GetElastic.com exclusively focuses on conversion rate optimization and a/b split testing for eCommerce websites. It’s a must read blog for anyone even remotely related to an eCommerce business.
You knew this was coming, didn’t you? I am very happy that Visual Website Optimizer Blog has seen explosive growth in readership and reach in last couple of months. This blog is not just about product/feature announcements only. Rather majority of posts relate to A/B testing case studies and conversion rate optimization how-tos. All time hits include: landing page optimization tips, impact of SEO on A/B testing, top 10 eCommerce sites (by conversion rate), etc.
Are there any other A/B testing resources that I missed? Please leave a comment below!
