When we saw the results Soocial had got from their latest A/B test, we were astonished! They added just two words next to the Sign up button and the conversions shot up by 28%. If we say the phrase was one of these: “Sign up for Free”, “It’s Free” or “Free Signup”, can you guess which one did the trick? That is precisely the beauty of A/B testing, you can never guess what works – they only way out is to actually test it. This case study has direct inputs from Soocial CEO, Stefan Fountain. He explains what they tested, why they tested and what others can learn from the results.
Background
Soocial is an online address book that helps you keep your phone, computer and online services contacts sane. The app syncs, merges and provides backups so that you are never stranded without the contact details you need. It works with over 500 phone models, webmail (like Gmail, Hotmail and Yahoo) and your Mac and Outlook. Soocial chose to use Visual Website Optimizer for their testing needs. Their first test was to increase the click-through rate on the homepage to the signup form.
What was tested
Their homepage features a large Sign up button. They wanted to test the age old principle of adding “free” to the call-to-action trigger. They tested a number of variations with different buttons including the word free, different colors and adding text next to the button. Part of this test was to see if correctness would trump brevity. For example one of the tested items was “Free up to 250 contacts” and another was “It’s free”. The former being technically more correct and the second being shorter but less “correct”.
Results
As you can read above, the changes on the page were extremely minor and, on the surface of it, look quite trivial. There is no reason why “It’s free” should work better than “Sign up for free”. Yet, it did! Out of several combinations, see the screenshot of the original version (control) and the winning variation.
Control: 14.5% conversion rate
Variation: 18.6% conversion rate
The only difference between winning variation and the original design is presence of “It’s free!” along side. And those two words increased the conversions by 28% (from 14.5% to 18.6%). This result was statistically significant and surely indicates that playing with the homepage really paid well for Soocial.
Why “It’s Free” Worked
When we asked Soocial why they thought the winning variation worked, this is what they had to say:
Being the nerds that we are creating Soocial, we thought that the most “correct” version would get the highest conversion. Of course the reason to use VWO is to confirm or deny our hypotheses and we will be testing a lot more variations in the coming weeks. Our hypothesis on the winning combination is that it doesn’t require any thought what-so-ever from users that it’s a low risk solution.
So, they believed that projecting the service as a low risk one did the wonder. We followed up the question asking what lessons can be derived from the test. Here is what they had to say:
To be honest we aren’t sure yet [of what impact this makes to the actual signups]. We wonder how these tests measure up to the goal funnels in Google Analytics and compared to actual conversions into paying customers. It could be that we get more users to signup and lose them later in the process. That is of course still valuable because it means we can have more call to actions to convert to a paying customer and test the conversion there.
For their next test, we recommend them to use VWO’s multiple goals functionality or VWO’s integration with Google Analytics to track the effect of variations throughout the funnel.
How valuable was Visual Website Optimizer for the A/B Test?
Here is what Stefan Fountain, CEO of Soocial, thinks about Visual Website Optimizer:
Invaluable. The ease and speed of setting up the test is brilliant and we can wait to start seeing the results on pages behind our login [a new VWO feature] to test on converting users to premium accounts. It will be very interesting to see the test results merely by changing the wording, graphics and positioning.
Key Takeaways
Two words can probably increase conversions for you as well. But the problem is that you don’t know which two words will work. Only an A/B test can answer that!
Since the day we released Visual Website Optimizer in beta, we have been seeking feedback on how to bring the tool closer to actual real-world needs. Till now, every VWO account was associated with a single login and it was a hassle to share that login across different people who are involved in setting up tests. Moreover, agencies and consultants wanted separation between different client accounts.
We are happy to announce that all these features have been launched thanks to inputs from our agency/consultant users, especially Peter from GetYouFound, Janco from Traffic4u and Jason from EmptyMind. Here is a rundown of the new features which are accessible under Account Management link in your account:
Create and manage permission based multiple-logins per account
For the same account you can create multiple different logins and can also restrict them to certain tasks only. Here is the screenshot of the login creation form:
Permissions are hierarchical in nature. Have a look at different options available:
Create and manage multiple (client) accounts
If you are running tests on multiple websites or running tests for multiple different clients, managing them can get quite cumbersome under a single account. Now you can create separate accounts for different clients/websites such that selecting an account only displays tests under that account. Have a look at the screenshot below:
Clicking on ‘Test Account’ will switch to that account and you will only see tests under that account.
Multiple logins for multiple accounts
Here is the killer feature: you can create an account for your client and then create new logins for that account only. What this means is that you can create Browse logins for a client and they will just be able to browse reports under their account (and all your other accounts will be hidden from them). This is a great way to share reports with clients without sharing all of your tests.
Of course, once you are done with a project/client, you can also deactivate their account which disables all logins associated with it.
How to access these cool features?
To repeat, you can access these new features under Account Management link in your account or click the following icon on your dashboard:
Feedback, Feedback, Feedback
We cannot operate in the dark and thus require feedback from you to make sure we are moving in the right direction. Try out the new feature, tell us if you like it or hate it. Tell us what modifications or new features you’d like. As you can guess, we are all ears for your feedback. So write us your viewpoint by leaving a comment here or emailing us at info@wingify.com. Excited to hear what you think.
AquaSoft was founded in 1999 and is a pioneer in photo presentation software. Their products cover a wide range from slide show creation, desktop publishing, photo books, etc. They recently candidly shared details from their latest A/B testing project, where they used Visual Website Optimizer for increasing sales from their eCommerce site by 20%. Here are the details on the test as told by AquaSoft:
Question: What was the conversion goal of the test?
Answer: The goal was to increase the sales on our product overview page. This page already had a higher conversion rate than the rest of our pages because people who visit this page already are interested in buying.
Question: On which page did you run the test?
Answer: Shop Overview (listing of all articles).
Question: Which part of page did you select for the test and what variations did you test?
Answer:
Original version (click to expand)
Redesigned version (click to expand)
Question: Why did you think that the variations you created had better chances to beat the original? What were you actually testing in this test?
Answer:
Note that we did not add any new feature nor the guarantee was new. It was just not described on the old page.
Question: What results did you get? Were you surprised by the results?
Answer: Improvements started with about 50% but settled on about 35-40% after some time. I was very happy with the result but didn’t trust it. What was measured by VWO were the clicks into the shopping cart, not the actual sales.
So I measured the sales with the funnel visualization of Google Analytics which showed an increase of 17.7% of actual sales. It also proved that the number of clicks into the shopping cart were about the same what VWO had measured. It shows that the bounce rate within the shopping cart was somewhat higher in the winning combination but the overall increased click rate outtakes the bounce rate.
In the second phase, we removed some whitespace and moved the product name above the box shots and gained another 10% increase in clicks and the final increase in sales settled at about 20%.
Question: Any lessons which can be derived from your test?
Answer: Don’t think your website is good as it is. Always test, always improve. It’s a slow process but worth doing.
Our concrete test shows that a clear modern design improve sales. Especially in the sales process trust building is very important.
Question: How valuable was Visual Website Optimizer for this test?
Answer: Is was so incredible easy to create the test (A/B split test in this case) that it takes nearly no time. Because we use an external shopping cart we can not measure in the cart itself so the feature of VWO tracking a click on a link without loading the page made it possible to run the test at all.
