Posted in How To on February 28th, 2013
Heatmaps within Visual Website Optimizer are great for getting an overview of how visitors behave on a landing page or website, but I frequently see that people don’t really use them to make a decision on anything. That’s unfortunate, because analytics and information should support decision making, and not just “provide an overview”.
To show how heatmaps are very useful to understand your visitors, I’ll discuss two case studies where they were the basis of successful A/B tests and then ask for your vote on one about Visual Website Optimizer.
Case Study 1: Google said DON’T put it on the left, but they still did
Dennis Publishing felt that they could earn more revenue from the Google ads on their website Carbuyer.co.uk. They saw on their eye-tracking heatmaps that the left pane of the website got a lot of attention from users. So they decided to shift the Google ads placement from under the review text to the left pane, in spite of Google recommending not to put it on the left.
Here is a comparison of the control and variation screenshots (click image to see larger version)
The in-house team at Dennis Publishing ran the test on 8000 visitors where it increased click through rate (CTR) by 74.5% (97% statistical significance). After making this test live site-wide, the CTR has increased by 44% and RPM (revenue per 1000 ad impressions) increased by 48%
That’s right. Dennis Publishing disregarded Google’s advice and chose to trust their own heatmaps to finally increase revenue by 48%
Case Study 2: Heatmap shows that the navigation bar is a distraction
Pair (an app for couples, now rebranded as Couple) had a simple landing page where visitors could go download the app or go to their Blog, Help and Jobs pages. This is the original page (click to see larger version)
Lim Cheng Soon, the growth hacker at Pair decided to run a test on the homepage using Visual Website Optimizer in which he only recorded the heatmap.
Lim explains what he did after that:
Turns out, I found out too many people click on the navigation bar on the top instead of clicking the conversion button (link to AppStore and Google Play). So I made up a theory that having too many “distractions” around the conversion button wasn’t such a good idea.
So I do a couple of A/B tests based on the theory of removing the “distractions” around the conversion button. Such as hiding the navigation, hiding the social sharing button, hiding the “Download for Free” text. I even do “counter theory” test as to prove my theory wrong by having a larger “Download for Free” text.
And turns out, my theory was almost correct:
- Hiding “Download for Free” text above the button gave a 10% increase of conversion.
- Hiding navigation menu gave a 12% increase of conversion.
- Hiding social sharing icons (facebook, twitter) wasn’t so good, a 34% decrease in conversion.
- Having a larger “Download for Free” text (my counter theory) gave a 42% decrease in conversion.
Based on the result of A/B testing, I did a new version of a homepage for split testing by moving navigation menu to the footer (instead of header), hiding the “Download for Free” text, but keeping the social sharing icon intact. And the result? 25% increase in conversion. Win!
This is what the variation page looked after all the tests (click on image to see larger version)
Three important lessons we learnt from this test
- Even though the menu may be difficult to see, don’t assume. Check your analytics and heatmaps.
- Clearing up the clutter around Call To Actions is worth testing.
- Social proof (if you have it in substantial numbers) goes a long way.
Case Study 3: Our turn! (we need your opinion on this one)
Okay so this isn’t exactly a case study. It is us asking for your opinion. Checkout the heatmap of our homepage (click on image to see larger version)
Do you see the problem? Visitors click on the “Free Trial” button on the top right more than they do on our primary call to action just below the headline, in fact, almost thrice as much. We think that we lose some conversions because the button is surrounded by other links that distract the visitor. To solve this issue, we have decided to conduct two A/B tests.
The first one is simple: Change the button text to something like “See how” or “Learn more” and direct to a page that explains A/B testing, talks about the benefits of using Visual Website Optimizer then asks visitors to sign up for a free trial. The hypothesis is that visitors need to know more before they sign up.
The second is redesigning the homepage to follow the F-shaped eye movement more closely.
Unfortunately, we’re strapped for time & resources and can only run one test at a time. Which one do you recommend we try first? (You’ll be voting through a tweet)
Click to vote for Test 1 – Change the primary CTA to direct to a page that explains A/B testing and Visual Website Optimizer
Click to vote for Test 2 – Add a video or image on the left and shift the headlines and CTA to the right
If you feel there’s something else we should do, let us know in the comments.
More optimization awesomeness
Know all that you need to get started:
- What is A/B Testing?
- Is it compatible with SEO?
- How to start your first A/B test?