It is interesting how counter-intuitive A/B testing results can get. One of the best examples of that counter-intuitiveness is a recent A/B split test. This test was done by a Visual Website Optimizer customer Vendio, an Alibaba.com company that specializes in providing free e-Commerce stores to its merchants. They have special landing pages for the free store signup and the objective of this test was to increase signup conversion rate on one of those landing pages.
Their original landing page (control) combined marketing content and registration fields in an attempt to reduce the number of clicks for a successful registration. This is how it looked:
Original Landing Page (with embedded signup form)
Note that it uses the so-called “best-practice” of embedding the signup form in the landing page itself. They had long been using the layout of the original page because as a best practice they presumed that reducing the number of clicks for a registration increased conversion rate. Although the page was performing relatively well, they wanted to make sure the included registration fields weren’t too aggressive or limiting in any manner.
So, the variation that they tested had somewhat similar imagery and content but the page didn’t include any registration fields and had slightly different styling. Clicking on ‘Signup Now’ button simply took the visitor to a page with signup form. In other words, Vendio added an extra step in their conversion funnel. Not a smart move, huh? This is how variation looked like:
Variation (no signup form) – 60% increase in signups!
Guess what? The page without the registration fields performed better – much better – to the tune of a 60% increase in conversions! Here is what Vendio had to say about the results:
Best practices are NOT always true! It’s still hard to believe, but the numbers don’t lie.
So, that’s the biggest lesson here and it is worth repeating: “Best practices are NOT always true”. If you do changes on your website or landing pages without A/B testing them, you are actually flying in the dark. Another lesson here is that it is it is worth testing radically different ideas – which, on the first glance, may appear not-so-smart (like removing signup form from the landing page!).
After the successful test, Vendio gave us a great testimonial:
Visual Website Optimizer was extraordinarily valuable. It was far easier to use than other solutions (some of which we couldn’t even get to work properly after implementation). Without VWO we would still be missing out on all those registrations, and we are continuing to see great and other surprising results with VWO.
Let’s admit it: getting ideas for A/B and multivariate tests is a hard job! It becomes even harder because of your familiarity with your website or landing page. Since you breathe-and-eat your website every day and know all its intricacies, it becomes incredibly hard to think of improvements. This phenomenon even has a medical name: Blind Spot.
To get ideas on what you can test in your next A/B or multivariate test, we have a A/B testing case study search engine called Ideafox. Also, earlier on this blog I talked about a service called Feedback Army which lets you have feedback from 10 people for $10 (I see the price is $15 now). Feedback Army is great service and we had received lots of relevant feedback from that site (in addition to some actionable ideas for A/B testing). As also noted in original post, the only drawback of this service is that people who give feedback are hired from Mechanical Turk, so feedback length is short and doesn’t go into too much depth (since those people are only paid 50 cents per feedback). Inspite of this, Feedback Army has a great ROI for an investment of $15.
What if you need feedback on your website from a professional?
Enter Concept Feedback, the website to get expert website evaluations and feedback. It is a community of design, usability and conversion lovers who comment and give feedback on your website and landing page. They have experts for three categories of feedback:
Experts have years of experience in their respective fields along with professional degrees in design. So, you can be guaranteed to get top-notch feedback on your website from Concept Feedback. Of course, all this comes at a price. You need to shell out $100 per expert feedback. So if you want feedback from 5 experts, that is going to cost you $500. It’s a little expensive but as you will read below, the quality and amount of feedback that you get from an expert has way more value than $100.
Evaluation of Visual Website Optimizer website on Concept Feedback
To try out the service, I evaluated Concept Feedback to get feedback on Visual Website Optimizer website design. Oh boy, I was pleasantly surprised with the quality of feedback on as many five different pages of our site: homepage, signup page, features page, case study page and blog. Here is some of the feedback produced verbatim:
All these comments come with a screenshot of your page so you can see where exactly the expert is referring to when he says “make your headlines sans-serif” (in this case, he was talking about our blog).
What’s more? The expert even quickly created a mockup of what according to him our site should look like. Here is the version he redesigned:
I think this redesign is definitely an improvement over existing design. What do you think?
Conclusion
For getting ideas for A/B and multivariate tests, nothing works better than feedback and evaluation services such as Feedback Army and Concept Feedback. The ‘strategy’ feedback type in Concept Feedback is especially relevant to A/B tests because there you have access to experts who have experience and knowledge in conversion rate optimization.
So, next time you are stuck and can’t think of anything to improve on your site, you know where to get feedback from.
We are proud to announce the integration of Visual Website Optimizer (VWO) with Unbounce, which is a tool for creating landing pages. If you are a VWO user and want to create a new landing page, you now know the tool we’d recommend.
The integration of the two tools is dead-simple and Unbounce has put together a post announcing why they recommend Visual Website Optimizer for multivariate testing.
Benefits of Integration
The main benefit, of course, is that using Unbounce you can effortlessly create new landing pages. So, if you want to start a new PPC campaign, for example, you won’t have to wait for your IT team to help you with that. Simply fire up Unbounce, create a new landing page, integrate it with Visual Website Optimizer and start optimizing it.
The main benefits of using Visual Website Optimizer with Unbounce is that you would be able to do the following for your landing pages created in Unbounce:
Read more about Visual Website Optimizer + Unbounce and watch an instructional video.
