The Next Generation of Visual Website Optimizer is launching on 29th April 2014 See What's Coming

Surprise, surprise! Having no secure icon on a page increased conversions by 400%

Posted in News on September 8th, 2010

Last week, we ran a contest (with $3000 worth prize) where we asked you to predict which variation in an A/B test produced 400% increase in conversions. We received over 50 entries and (ironically) the split of answers was roughly 50/50. This shows just how hard it is to predict A/B test winner in advance. Only real data tells the truth!

For this case, now you have the answer: the variation that DID NOT have security badge actually increased conversions by 400%. This is in stark contrast to one of our earlier A/B testing case studies where a variation which had security badge increased conversions by 32% (over the control which did not have such badge). This shows that every site is unique and best practices shouldn’t be adopted without doing any testing to prove that they would work for your site too.

Out of all the correct entries, we’ve selected one entry randomly to win this contest:

The $3000 worth prize (twelve month Visual Website Optimizer $249/mo subscription) goes to: Subhash Surampudi [link to his comment]. Congratulations to the winner!

Case Study

ICoupon Blog is a coupon website that has been around for about 5 months and provides coupons and rebates for different services and products. They decided to use Visual Website Optimizer to increase click-throughs on these coupons.

What they tested was whether having a large ‘Secure’ icon in the sidebar would help with our conversion rate. They thought the icon would help because it would push down other (distracting) text in the sidebar. [Note to ICoupon: a great follow up test would be to further test this hypothesis by having a variation where sidebar text is completely removed]

Here are the variations they tested:

– Version A: with Security badge in sidebar –

– Version B: without Secure badge in sidebar –

Reasons why this particular variation was tested

In the words of Bradley Spencer (from ICoupon):

Actually I thought the ‘Secure’ icon would win hands down. It was my idea and I wanted to prove to my partners that it was a good idea. I was really surprised to learn that it didn’t help at all.

They also had some user testing done and everyone seemed to notice the ‘Secure’ button a lot. It was one of the most prominent parts of the page. So they think removing it helped focus the user on they we really wanted them to push.

A/B test results

They found that without the secure icon had over 400% improvement on conversions as compared to having the image. [Note: results are statistically significant]

Strangely, removing the icon dropped site engagement by about 17%. However, with their business model a ‘bounce’ is actually a good thing, so site engagement isn’t as large of a factor.

In nutshell, in this particular case, it looks like the secure icon distracted the visitor from the main focus of the page.

Lessons

As Bradley sums up the lessons for others:

Make each page designed to get the user to do one thing, and try to focus all of their attention on that one thing.

He also adds a word about the tool they used for the test:

Visual Website Optimizer was invaluable. We would have kept wasting user’s attention with that icon without VWO.

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


17 Comments
James Barnes
September 8, 2010

When providing my online payment details, the last thing I want to see is that your site security is a selling point. That should be a given.

In attempting to create trust perhaps you created disbelief and distrust?

Adam
September 8, 2010

Perhaps the results are due to the “secure” badge not being attributed to a 3rd party. To me, it’s just an image.

I wonder if the badge would have different results if it were third party secured. Credibility via outside party…

Interesting, nonetheless.

-Adam
@apzuckerman

zack
September 8, 2010

Why the hell did they push the search box down below the secure thing. That’s just stupid.

Mauricio Wolff
September 9, 2010

I agree with @apzuckerman. Do the same test with a verisign logo, for instance, and publish the results… ;)

A big custom image like that just make the customer suspicious. If there are so many trusted seals around, why make a huge statement with a worthless badge? Looks fake.

Regards

@bitbonsai

Matt Roche
September 11, 2010

As the comments above indicate, this article makes claims that are not real. Re-run the test with a trusted logo.

I can imagine that a “fake” trust logo is a massive disincentive. Brands work because they engage subconscious triggers. Verisign has developed a brand, it has significant repetition and reinforcement, and therefore its emblem stimulates a trust response.

The very unfamiliarity of your “secure” symbol, coupled with the explicit “trust me” message, makes it an anti-brand. It does not communicate the trustworthiness of the organization for which it is the brand, it triggers distrust.

A simple way to think about it – when someone (especially a stranger) says “trust me”, what does that signal to you? You can see how there is a different response from a trustworthy source saying “trust them”.

Carl Juneau
September 15, 2010

Great case study, thanks Paras.

I myself saw a decreased in sign-ups on my landing page when I tested thrust seals there.

I’m thinking we should keep those for checkout pages.

Best,

CJ

[...] A recent test was very surprising – in this test it was found out that removing a secure icon from the page [...]

Nick
October 6, 2010

Great little article, I’ve seen so many commerce sites that don’t even mention security. I wonder how much they’re losing in sales!

Michael Kjeldsen
October 6, 2010

I have to agree – that badge doesn’t invoke “security” and “trust” in me. It looks more like an advertisement or something of that sort…

Massage Pro Wanda
December 13, 2010

No one mentioned the fact that it’s a large dark green image which, of course draws your eye. I had trouble looking at anything but that big green thang!

[...] The funny thing is that once you start eyeing those performance metrics of yours you’ll end up getting insights about your traffic that you can never learn from anyone else. Business online is all about knowing your own audience and constantly improving your site. After all, a banner or button in the wrong place might kill your conversion or increase it by 400%. [...]

Alon
February 25, 2011

Maybe it was the size of the button that was the cause of this effect? Other than drawing attention away from the call to action button, perhaps it sent a wrong signal. If a website exclaims he’s safe, it might appear to some users as an indication that it’s really not.
How about testing conversions with a small secure logo in the footer?

Richard
March 18, 2011

Interesting stuff. For those who say it proves nothing because the badge wasn’t “official” you’re missing the point. What it showed was that design 2 delivered more than design 1 which would have remained unknown unless the test had been run. Not everyone is an expert on web design (and even the experts get it wrong from time to time) so choosing the right badge isn’t as obvious as we might think.

[...] The results of this A/B test and the winner of the contest have been announced! Congrats to Subhash Surampudi for winning the [...]

[...] So, next time you think about optimizing one of your pages, try to think about which element on page you can afford to remove. Perhaps it’s the large product screenshot or video that’s hurting the conversions? Or, perhaps it is the trust / secure logo that’s decreasing the conversions? [...]

Brian
July 7, 2011

Question:
What statistics do you use to track and determine “user engagement”?

Brian

Simon Potter
May 17, 2012

The reason it pulled better is this…having a big green button that says secure doesn’t make make it secure.

Nobody believed the badge was real…looks cartoonish and frankly looks like your trying to fools someone

Would wearing a dime store badge on my shirt make me sheriff…no more than that badge ACTUALLY makes your site secure

The badge is disingenuous and caused serious doubt…400% increase shows that

Leave a comment
Required
Required - not published


− four = 2

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com