UPDATE: The results of this A/B test and the winner of the contest have been announced! Congrats to Subhash Surampudi for winning the contest!
In the past we have published numerous A/B testing case studies which showed how even little changes could lead to dramatic improvements in sales, downloads, and and conversions. Today, we have a fresh case study where the user was able to achieve a whopping 400% increase in conversions. Yes, read that once again: a mind-boggling 400% increase in conversions (and that too statistically significant) with just a single change. We decided that from now onwards before publishing the full case study on the blog, we will run a teaser contest.
Challenge: Guess which version is the actual winner in this A/B test.
Prize: Out of all the correct entries, one lucky person will win a full year of $249/month Visual Website Optimizer subscription!! Twelve months of our Large Agency plan is total worth $3000 (12*249).
How to enter?: Simply leave a comment on blog and/or tweet about which variation you think increased conversions by 400%. If you tweet, either use the tweet boxes belows or make sure you cc @wingify so that we can track your entry.
When will be the winner chosen?: on September 8th, along with the actual case study, we will announce the winner of this contest.
About this A/B test case study: one variation had a Security badge in the sidebar and in the other there was no such badge. One of the variations saw an increase of 400% in conversion rate (click on coupon code).
So, which version increased conversions by 400%?
– Version A: with Security badge in sidebar –
– Version B: without Secure badge in sidebar –
Remember, the prize for this contest is a full year of $249/mo Visual Website Optimizer subscription total worth $3000! So, enter take part into contest now either by tweeting your choice above or leaving a comment below. (Bonus: if you do both, your entry will be counted twice.)
RSS feed for comments on this post. TrackBack URL
My guess would be version A: WITH the security-badge.
If, and when, it’s announced as the winner, I would try with a more official looking badge, as compared to this cartoon-ish one
Comment by Michael Kjeldsen — September 1, 2010 @ 7:43 pm
The one without the “Secure” badge gets more conversions. That “Secure” badge looks too generic and fake. It probably raises more red flags about it possibly not being secure.
Comment by Justin — September 1, 2010 @ 7:46 pm
My Bet is Version B
Comment by Richard — September 1, 2010 @ 8:29 pm
Version B
Comment by Damir — September 1, 2010 @ 8:40 pm
I’ll go with version B. The security shield takes the attention away from the call-to-action button.
Comment by Julie — September 1, 2010 @ 9:03 pm
Version A – With the security badge
People see the credit card logos and then instantly question the security of the website as to whether they should give you their credit card details. They then associate the security badge with the credit card logo and assume the website is safe to use.
Comment by Jamie — September 1, 2010 @ 9:10 pm
Version B
Comment by David Crowther — September 1, 2010 @ 9:11 pm
B, without the badge. A single, clear call to action that is the Coupon button. No distraction of the other big colorful thing in A.
Comment by Jim Jarrett — September 1, 2010 @ 9:20 pm
BTW, if you go look at the site, the Secure badge is on the home page but not the Coupon page. Doubt they’d keep a losing design up.
Comment by Jim Jarrett — September 1, 2010 @ 9:22 pm
Version B FTW!
Comment by Joe — September 1, 2010 @ 9:23 pm
My bet is on version A: with the security badge.
We’ve just run a few test ourselves and 3 out of 4 badges boosted our conversions. I’d guess version A increased conversions by 15%.
Comment by Jens Schriver — September 1, 2010 @ 9:54 pm
Usually it would be the one with the secure badge, but this one doesn’t look from a reputable firm.. Even still, I’ll go with version A.
Comment by wesley — September 1, 2010 @ 10:06 pm
I guess version B with no security badge increased conversions by 400%. Just a gut feeling.
Comment by Franz — September 2, 2010 @ 12:33 am
@Jim Jarrett
The tests are still running as both versions still appear!
Comment by Jamie — September 2, 2010 @ 12:43 am
I’d go with Version B. It has a clear single point of attention.
My suggestion for these teasers going forward, mention the objective for the test as well.
In this particular case, all of us are making an assumption.
Incidentally, shouldn’t you also include some information on how the traffic is being driven to the test page? If not in the teaser at least in the case study.
If I am driving traffic to the page being tested from a targeted (to the test objective) ad vs. from the home page (a more generic source), the results might be different.
Subhash
Comment by Subhash — September 2, 2010 @ 12:31 pm
@Subhash: great point. I think you may have missed but the objective of the test was to increase clicks on ‘Coupon Code’. Yes, source of traffic is important and we will definitely include that in future case studies and teasers. Targeted v/s causal traffic will respond to the same test differently; in this case it was more or less casual traffic.
Comment by Paras Chopra — September 2, 2010 @ 12:34 pm
My vote is B.
Not only is the badge large and distracting, but it actually can put fear in the users mind. Here is a customer’s potential thoughts: It’s just a coupon, so why should it need to be secure?
Comment by Robert Brady — September 2, 2010 @ 8:24 pm
I would be shocked if it wouldn’t be A. Customers always love secure purchase forms!
Comment by anty — September 3, 2010 @ 12:52 pm
VERSION B
A single clear call to action, without taking away the required attention.
The secure code looks “fake” and could get the visitor click the back button.
Pramodh
@pramodh
Comment by Pramodh — September 3, 2010 @ 4:10 pm
Version A
As a more or less casual visitor, I am concerned about the web pages that I visit. And if there are aspects of “trust” visible, it allows us to go further.
Vaishali Hukkeri
Comment by Vaishali Hukkeri — September 3, 2010 @ 4:12 pm
Version B – the badge graphic takes the focus off of the call to action.
Comment by David — September 3, 2010 @ 6:44 pm
I say . . . VERSION B
I am thinking the overall copy needs some work and wonder if the +400% is in the neighborhood of less than 20 total redeemed.
“Get $12.5 Cash Back Discount On Ambren” just deesn’t read or sound right, e.g. – wouldn’t it be $12.50 (with the zero).
So I think the clip-art looking badge just adds to the overall sense of risk around the whole offer.
Although, as I have said before, “What matters is what performs best . . . not personal preference.”
Comment by Chris Hopf — September 3, 2010 @ 9:45 pm
Version A
Comment by TaeWoo Kim — September 4, 2010 @ 2:02 am
Version B. It makes coupon code button in focus.
Comment by Girish — September 4, 2010 @ 11:59 am
Version A
Comment by Brian R, — September 4, 2010 @ 1:22 pm
My bet: a security badge version.
Comment by Melvin Ram — September 5, 2010 @ 6:03 am
I’m thinking A – with the badge
Comment by Kevin Dahlhausen — September 5, 2010 @ 12:39 pm
@All who participated: The results of this A/B test and the winner of the contest have been announced! Congrats to Subhash Surampudi for winning the contest!
Here is the post: http://visualwebsiteoptimizer.com/split-testing-blog/surprise-surprise-having-no-secure-icon-on-a-page-increased-conversions-by-400/
Comment by Paras Chopra — September 8, 2010 @ 2:02 pm