Many people believe that A/B testing goes against the basic sentiment behind usability. After all, a usable website should have high conversion rate and so you don’t require A/B testing? Well, answer to that question is both yes and no. Every highly converting website should be usable, but every usable website need not be highly converting. That is because the same website can be made usable in multiple different ways each of which may have different conversion rate. Finding what set of usability techniques work best for the website or landing page is what you need A/B split testing for.
In this particular case study (also featured in WhichTestWon), you will see how following a usability “best-practice” (of having a navigation menu) increased conversions by 34% for Slideshop.com, an eCommerce website selling powerpoint templates. This test was conducted by our certified partner agency Optimeria. They specialize in using Visual Website Optimizer for conversion rate optimization with a tight focus on analysis of browsing behavior and psychology of customers v/s non-customers.
The goal of this A/B test was to create increase user engagement (on site), and also create a better flow from categories to subcategories and between main categories. In original version of Slideshop.com, all pages had a promotional bar on the right which had special offer, customer testimonials, etc. See how original version looked like:
Original design of Slideshop.com
As is true for all successful A/B tests, Optimeria didn’t randomly come up with variations to test. In fact, they analyzed the browsing behaviour of the visitors, and saw that they weren’t as committed to staying within a category or subcategory, and heatmap and clickflow analysis of multiple key pages indicated that visitors did not sufficiently engage the top menu – specifically subcategories. This let to the thesis that by improving engagement we would see more activity in the more focused subcategories. The aim was to also improve usability on the site, and see how it affects clickflow patterns.
Optimeria decided to test a version with better usability. They replaced the promotional right sidebar by a left navigation bar that did not have the heavy focus on single visual elements (VWO team helped them setup this test, see details of how left v/s right A/B test was setup). See below how variation with left navigation menu bar looked like:
Navigation menu on the left: 34% increase in Add to Cart clicks
Note that all subcategory pages are neatly displayed in left navigation bar and it is much more intuitive to browse around different pages on the ecommerce store.
The variation design for Slideshop saw 8.9% lift in user engagement and a staggering 34% lift in people adding products to cart. Besides that a lot of key patterns emerged when looking at the new clickflow patterns. They were impressed with the engagement lift, as Slidshop already had a good engagement rate, but they was very surprised about the lift in adding to cart (as it was not expected – they simply added a navigation bar, why should clicks on add to cart increase?).
The increase in conversion rate, of course, made Optimeria and Slideshop.com very happy. Karsten (of Optimeria) gives a lot of credit for that success to tool used for testing:
What I like the most about Visual Website Optimizer is specially the fact it saves me a lot of time in the actual execution of the test -phase, so that I can focus a lot more on the hard analysis, thesis and reporting phase. My clients don’t necessarily see the difference, but they get much more learning value from me using VWO because I can focus on what I do best.
Also when I am dealing with a client that has an overstressed IT department (Slideshop does not, but it happens more often than not) it’s great that I can plan, setup and execute tests and deliver results for decision makers without having to rely on a sometimes slow and tough IT process.
Decision makers often like the fact they can get feasible results before they need to involve IT department in doing changes to the website.
When asked for the lessons learnt, Karsten Lund (CEO of Optimeria) said:
One should try to minimize browsing obstacles for visitors. Sometimes more clicks are better as long as they indicate a move forward in the conversion funnel. Better usability creates more motivated visitors.
The counter-intuitive idea of “more clicks may lead to higher conversion rate” was also proved in another A/B testing case study where it was found out that removing a signup form on landing page actually increased signups by 60%.
Seems like A/B testing usability techniques is an important part of conversion rate optimization!
I just published a guest post on Smashing Magazine titled Multivariate Testing in Action: Five Simple Steps to Increase Conversion Rates.
Essentially, there are five steps to increasing conversion rate:
If these steps sound complicated to you, I recommend you to read the extensive tutorial which has numerous examples.
My article explains multivariate testing by means of a case study where I tested following variations on a software download page (notice color and text changes):
Can you guess which variation produced maximum downloads? Well, the end result of this test was that #10 combination (in the screenshot above, one with ‘Download for Free’ in red) had 60% improvement in conversion rate. That’s the power of multivariate testing.
Read the full case study and tutorial: Multivariate Testing in Action: Five Simple Steps to Increase Conversion Rates. I hope you like it!
The Nielson Company regularly releases data for conversion rate of different websites via its MegaView Retail service. The data (as of March 2010) for top 10 online retails by conversion rate is analyzed in this post. The objective of this list is to give you an idea about to what extent these retailers have been able to optimize their conversion rates.
Note the eligibility criteria for this list: To be considered, e-commerce sites must have had a minimum of 500K unique visitors during the month. Conversion-rate data is based on visitor conversion rates, not session conversion rates: i.e., No. of unique customers/No. of unique visitors.
Schwan’s: 40.6% conversion rate
Schwans.com is an online grocery store and food delivery business. A conversion rate of 40.6% sounds too good to be real but Schwan’s is an old company which started home deliveries of food in 1952. So, it has a good brand name as far as food delivery is concerned and now most of its customers are simply visiting the website to order the food they want. No wonder, conversion rate is so high!
Woman Within: 25.3% conversion rate
Woman Within offers Plus Sized clothing for women. Note their laser-sharp focus: “plus sized clothing for women” which (combined with large collection and clean site design) is probably the reason for high conversion rate.
Blair.com: 20.4% conversion rate
Blair.com sells clothing online. Just like Schwan’s, Blair.com is also a very old business (started in 1910!). So it has brand and customer base already established and the website serves as a channel for ordering.
1800petmeds.com: 17.7% conversion rate
As its name says, 1800petmeds.com sells pet medication. This eCommerce store is a perfect example of focusing on a niche. I bet they would never see such high conversion rate, if they attempted to sell everything related to pets (food, clothes, grooming products, etc.) The focus of store is pet medication and that’s the reason conversion rate is high. Also note that they have an extensive mini-site on Pet Health education, which probably drives them numerous targeted prospective customers.
Vitacost.com: 16.4% conversion rate
Online store selling vitamin supplies. I love the clean sidebar on the left. The website is bit old-schooled and is optimized for IE (that too for a lower screen resolution). They probably get a lot of non-tech savvy visitors so the sidebar would be very helpful for product selection. This shows how focusing on the customer (and not on the design) can help increase your conversion rates.
The rest 5 ecommerce sites (by conversion rate)
Secret of high conversion rates
Of course, there is no secret for such high conversion rates. These companies have been doing conversion rate optimization for years now and one cannot hope to replicate the same overnight. However, looking at the websites above, I can imagine a few general trends on why they have such high conversion rate:
Hope you liked the examples and my analysis. Please leave a comment below if you have any other examples of high converting websites.
It is interesting how counter-intuitive A/B testing results can get. One of the best examples of that counter-intuitiveness is a recent A/B split test. This test was done by a Visual Website Optimizer customer Vendio, an Alibaba.com company that specializes in providing free e-Commerce stores to its merchants. They have special landing pages for the free store signup and the objective of this test was to increase signup conversion rate on one of those landing pages.
Their original landing page (control) combined marketing content and registration fields in an attempt to reduce the number of clicks for a successful registration. This is how it looked:
Original Landing Page (with embedded signup form)
Note that it uses the so-called “best-practice” of embedding the signup form in the landing page itself. They had long been using the layout of the original page because as a best practice they presumed that reducing the number of clicks for a registration increased conversion rate. Although the page was performing relatively well, they wanted to make sure the included registration fields weren’t too aggressive or limiting in any manner.
So, the variation that they tested had somewhat similar imagery and content but the page didn’t include any registration fields and had slightly different styling. Clicking on ‘Signup Now’ button simply took the visitor to a page with signup form. In other words, Vendio added an extra step in their conversion funnel. Not a smart move, huh? This is how variation looked like:
Variation (no signup form) – 60% increase in signups!
Guess what? The page without the registration fields performed better – much better – to the tune of a 60% increase in conversions! Here is what Vendio had to say about the results:
Best practices are NOT always true! It’s still hard to believe, but the numbers don’t lie.
So, that’s the biggest lesson here and it is worth repeating: “Best practices are NOT always true”. If you do changes on your website or landing pages without A/B testing them, you are actually flying in the dark. Another lesson here is that it is it is worth testing radically different ideas – which, on the first glance, may appear not-so-smart (like removing signup form from the landing page!).
After the successful test, Vendio gave us a great testimonial:
Visual Website Optimizer was extraordinarily valuable. It was far easier to use than other solutions (some of which we couldn’t even get to work properly after implementation). Without VWO we would still be missing out on all those registrations, and we are continuing to see great and other surprising results with VWO.
Let’s admit it: getting ideas for A/B and multivariate tests is a hard job! It becomes even harder because of your familiarity with your website or landing page. Since you breathe-and-eat your website every day and know all its intricacies, it becomes incredibly hard to think of improvements. This phenomenon even has a medical name: Blind Spot.
To get ideas on what you can test in your next A/B or multivariate test, we have a A/B testing case study search engine called Ideafox. Also, earlier on this blog I talked about a service called Feedback Army which lets you have feedback from 10 people for $10 (I see the price is $15 now). Feedback Army is great service and we had received lots of relevant feedback from that site (in addition to some actionable ideas for A/B testing). As also noted in original post, the only drawback of this service is that people who give feedback are hired from Mechanical Turk, so feedback length is short and doesn’t go into too much depth (since those people are only paid 50 cents per feedback). Inspite of this, Feedback Army has a great ROI for an investment of $15.
What if you need feedback on your website from a professional?
Enter Concept Feedback, the website to get expert website evaluations and feedback. It is a community of design, usability and conversion lovers who comment and give feedback on your website and landing page. They have experts for three categories of feedback:
Experts have years of experience in their respective fields along with professional degrees in design. So, you can be guaranteed to get top-notch feedback on your website from Concept Feedback. Of course, all this comes at a price. You need to shell out $100 per expert feedback. So if you want feedback from 5 experts, that is going to cost you $500. It’s a little expensive but as you will read below, the quality and amount of feedback that you get from an expert has way more value than $100.
Evaluation of Visual Website Optimizer website on Concept Feedback
To try out the service, I evaluated Concept Feedback to get feedback on Visual Website Optimizer website design. Oh boy, I was pleasantly surprised with the quality of feedback on as many five different pages of our site: homepage, signup page, features page, case study page and blog. Here is some of the feedback produced verbatim:
All these comments come with a screenshot of your page so you can see where exactly the expert is referring to when he says “make your headlines sans-serif” (in this case, he was talking about our blog).
What’s more? The expert even quickly created a mockup of what according to him our site should look like. Here is the version he redesigned:
I think this redesign is definitely an improvement over existing design. What do you think?
Conclusion
For getting ideas for A/B and multivariate tests, nothing works better than feedback and evaluation services such as Feedback Army and Concept Feedback. The ‘strategy’ feedback type in Concept Feedback is especially relevant to A/B tests because there you have access to experts who have experience and knowledge in conversion rate optimization.
So, next time you are stuck and can’t think of anything to improve on your site, you know where to get feedback from.
Visual Website Optimizer has a feature that allows running of same A/B or multivariate test on multiple pages at once. For example, if your site has several product categories and you would like to run a test only on one category, all you need to do is to enter a URL pattern like http://example.com/shoes/* in Visual Website Optimizer.
The URL pattern above will make the test run on all pages in the shoes directory. This feature is very handy because you can test common call-to-actions, headers, footers, sidebars, etc. consistently on multiple pages at one go. What’s more, a consistent variation is shown on all the pages in a test. So if a user sees red variation on one shoes page, he will always see the same red variation on all other shoes pages.
Sometimes (primarily due to SEO reasons) there is no clear pattern in the URLs. That is, it may be possible that there are several independent URLs for category or product pages without any apparent pattern. For example, if there are URLs such as http://example.com/adidas/, http://example.com/reebok/, http://example.com/microsoft/, http://example.com/adobe/, etc. and you wish to run the test on only on shoes pages (the first two URLs), how are you going to do that? Until now it wasn’t possible to setup such a test, but with our new feature Custom URLs it is very simple to setup such tests.
Actually, there are three easy steps if you want to use custom URLs:
On all pages where you need to run a specific test, output _vis_opt_url variable and modify VWO code as following (bold parts are extra modifications):
<head> ... // Only on pages where this test is to be run var _vis_opt_url = 'http://example.com/my_shoes_pages'; ... ... ... // Common VWO Code var _vis_opt_url = typeof(_vis_opt_url)=="undefined" ? document.URL : _vis_opt_url; .... '&url='+encodeURIComponent(_vis_opt_url) +'&random='+Math.random() ... </head>
We do realize that the implementation is bit complex but we wanted to roll out the feature to all customers (many enterprise customers have been using it already). We are here to help you implement VWO code for custom URLs. Simply contact support@wingify.com if you need our help and assistance.
We are proud to announce a Certified Partner program for agencies / consultants who have been using Visual Website Optimizer to optimize their clients’ conversion rate and sales. We have a host of benefits for the agencies that are accepted into the program:
In a nutshell, if you are accepted into this program, you will enter into an exclusive club with a world of benefits. Best of all: there is no cost for applying to partner program. However, the space is limited (initially, we are planning to accept only 30 agencies into the program) and you must meet certain eligibility requirements:
How to apply?
Simply email us at partners@wingify.com with your Visual Website Optimizer account username and any other information needed to satisfy eligibility requirement. We will notify you within one week if you are accepted into the program.
NOTE: Initially, we are planning to accept only first 30 eligible agencies. So, if you plan to become a Visual Website Optimizer certified partner, you must apply soon.
Last week I offered free conversion rate optimization advice on a popular forum (known as Hacker News). Within a single day I got 50+ requests for help. It was definitely an enriching experience analyzing and dissecting all those websites and landing pages. As I was replying and providing my feedback to those 50+ sites, I started sensing a few common issues that affected conversion rates of all those pages. Without naming any specific URL or site, in this post I will detail where those sites lacked and what you can learn from my analysis to fix your homepage or landing page.
According to my analysis, one of these four issues (if not all) were the most common causes of poor conversion rates. As you go through the list of issues below, try to relate them to your landing page (or homepage). Here are the four most common issues that affect conversion rate and sales:
Let’s analyze the issues one by one.
Example of a homepage with too much text
I won’t be surprised about lack of sales and conversions if your page announces “Welcome to..” followed by a bunch of three paragraphs describing what the site is about. Nobody on the web has patience to read paragraphs after paragraphs about you are offering.
On the web, people scan for elements that catch their eye. And you get only first few seconds to answer two most important questions: a) what you are offering; b) why they may need it.
Example of a balanced webpage (text + images)
So, what you need is a proper balance between graphics and text. Some examples of how you can improve your landing page (by replacing some text):
Hiring a professional web designer will certainly help if you lack design skills. On a similar note, “visually appealing” pages are always seen as more credible than “crudely designed” pages. So, an investment in a good design will go a long way helping your sales and conversions.
Example of confusing headline: “Changing how the world works”
As I hinted in the section above, visitors on your page are impatient. Within first 5 seconds, they want to know what your service does or you have lost a chance with him/her. As someone wise said: “Browser back button is your biggest enemy” (if you find the source of this quotation, please leave a comment). Never think that a visitor is going to spend minutes reading through all text on your page and then make his best guess of what you are offering. Instead, you should make the job easy for him. Have a big, bold descriptive headline as the first thing he should see.
Example of good headline: “Hire Online Workers to get the Job Done”
A descriptive headline also serves another important job: it *sticks* in the visitor’s brain as long as he stays on your website. Contrast this to the scenario where there is no helpful headline which a visitor can fall back on if your page gets too confusing (usually happens because we want to write about EVERY feature our site offers). Moreover, your visitor is usually distracted. Imagine a “busy-beaver” visitor chatting with friends on IM, doing a status update on Facebook/Twitter and on a call with his boss, all at once. Now imagine he stumbles on your website. Do you expect him to really understand what your site does without having a descriptive headline?
My advice is to avoid following kinds of headlines:
In a nutshell, headlines should be short, concise and descriptive.
Paradox of choice on a landing page. Out of three call-to-action buttons. which option to choose?
Call-to-action is a button or link that asks visitor to take a specific action. It may be a link to your signup form, plans and pricing page or the feature tour page. There are two specific issues related to call-to-action: a) either some sites don’t have any call to action button or b) some sites have too many call-to-action buttons. Once the visitor arrives on your page, thinks that you are credible (from your design), reads the descriptive headline and is finally convinced to spend some time on your site, what’s the next page you want him to see? That decision should not be left on visitor because only you know (and not him) which is the most relevant page that the visitor should be viewing next.
Example of single, prominent call-to-action: “Download Wordpress”
If you don’t have a single call-to-action or have far too many call-to-action, visitor is likely to get confused what to next (since all links from your landing page/homepage seem to be of equal importance). Even if you have two prominent buttons (e.g. one of Learn More and other for the signup), try reducing it to one button. There is even a book titled: “Don’t make me think!” and that’s precisely the point I’m trying to make here. Don’t force your visitor to make a choice. By placing relevant call-to-action buttons on different pages of your site, you should gently guide him to the final goal (be it a signup, purchase, download, etc.)
Example of no social proof. Why should I bother about Twhirl?
So you make bold claims on your site. Of course, you think you are the “Best Twitter client ever”. But, unfortunately, making claims is easy. Any site can claim to be the “best” or “revolutionary” because those words are abstract. You may think your product is the best but if you are the only one in this world with that viewpoint, you are not going to convince anyone to try it out.
Humans crave for social proof. They want to know whole else is using this thing and how beneficial was it for them. Even if you design the most beautiful landing page but fail to include any social proof, your sales and conversions are going to suffer. Social proof can be shown in terms of testimonials, company logos, customer photos or case studies.
Example of social proof: we know Facebook, LA Times, etc. use Hootsuite. So it must be good, no?
It is understandable that if your site is just getting started, it may be hard to get any social proof because you may not have any customers. In that case, you need to have a convincing return-on-investment proof on your site. I’m not just talking about justifying investment of money but you also need to convince a visitor to invest time trying out your service or product. People crave for statistics and validation. So, you can perhaps do a small study or research on Internet to come up with metric of some kind highlighting usefulness of your service. (Example if you have a new social media monitoring service: 95% of business are talked about on the Internet, use MyShinyNewTool to talk to those invisible customers).
Another key point with regards to social proof is human emotions. People respond to concrete representations (say a customer video testimonial) in a much engaging way as compared to an abstract fact (say, 50+ companies from Life Sciences and Biotechnology industry use our software). This is not to say that facts in your social proof don’t work. They do. But you can always augment them with stories of individual customers and what your service did to them. (Case studies are a great way about doing that).
To re-iterate, if you want to increase sales and conversions on your landing page or homepage, you need to concentrate on fixing following issues:
We also have a FREE automated diagnostic tool, called Landing Page Analyzer, which will point these (and many more) deficiencies on your page. Click here to start analyzing your landing pages.
