The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

Debunking the Social Proof myth

Posted in A/B Split Testing, Case Studies on November 21st, 2012

Social media and how it can be used in digital marketing has been the subject of many articles and is a centerpiece of many digital strategies. However, recently conversion testing has shown that social media can actually harm some websites, but has not yet clearly established why and how to avoid this ‘social media conversion trap’.

CalPont is a database management software company based in Frisco, TX. CalPont makes InfiniDB, an award winning, scalable, software-only columnar database for big data analytics applications. CalPont engaged Hathway – a digital agency in California with expertise in strategy and development for mobile and web experiences (yup, I work here) to help establish CalPont’s web presence and optimize the website for end users and search engines.

So, after reviewing the social media implementation on the CalPont website, which appears in three areas, Hathway determined that a conversion test was needed to establish if the social media implementation was working:

  • In the left side below three major Call To Action buttons
  • On all backpages at the top of the content
  • In the footer

CalPont – Without social sharing features

CalPont – With social sharing features

There are two major Social Media integrations; one allowing users to ‘Share’ the content on their own social media profiles, and another where users can ‘follow’ a company on a social network. CalPont only uses the ‘share’ or ‘like’ functionality in all three instances.

Paying particular attention to the icon set located at the top of all content, which may act as a distraction to users and interfere with the informational aspect of the site, not to mention the conversion purpose. In addition, the displayed ‘Likes’ are often below 5, thus failing to provide social proof and gaining users’ trust. Thus the hypothesis for this test was, that removing the social media icons from the top of the page content, would help conversions because users no longer saw how few people ‘liked’ the page, and because the content was given more weight and focus because it was higher up on the page and there was less distraction.

Thus an A/B test was designed that removes the social icons at the top of all pages, leaving only the Sidebar and footer sets. This variant was then served to 50% of the website’s audience (randomly chosen) and 11 goals were measured against both variants. The test ran for 3 weeks at which point three goals had reached full statistical conclusion showing big wins and many of the other goals were also showing positive trends.

Only two goals showed a decline: Debian Downloads and Engagement. The slight decrease in engagement suggests that the primary problem with this social network integration was the prominent placement, the type or choice of networks as well as the low ‘likes’ rather than the distraction factor. The main goal, ‘Sale Inquiry SENT’ which measures users who have successfully submitted a Form to buy the product, showed a massive gain, with 96% statistical confidence.

Why does this happen?

In  their post “Sweep The Sleaze“, Information Architects make a solid case for not including social media buttons on your website, specially when it hinders real communication:

If readers are too lazy to copy and paste the URL, and write a few words about your content, then it is not because you lack these magical buttons.

or

If you’re unknown, social media buttons make you look like a dog waiting for the crumbs from the table. You might have magnificent writing skills and a lot to say, but you will still only get a few retweets and likes. Yes, it’s not fair, but that’s the way it is. If you’re known, you will get attention, even for the mediocre. If you’re not known, no matter how good you are, initially you won’t. That button that says “2 retweets” will be read as: “This is not so great, but please read it anyway? Please?”

and

We removed FB buttons and traffic from Facebook increased. Reason: instead of "liking" articles, readers share it on their timeleine.
@smashingmag
Smashing Magazine

But finally, here’s another bit of A/B testing that should make you think: How AMD used A/B testing to achieve 3600% increase in social sharing.

Anne Stahl

As a UI / Front End Developer, my keen interest in best practices coalesced with my background in human behavior and science, and I joined HATHWAY, a digital agency in Southern California, as their digital strategy manager, where I lead a small team producing user tests to increase conversions.

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

14 Comments
Laura
November 21, 2012

Hmmm, I’m not sure I agree with what Information Architects when they say,

“If readers are too lazy to copy and paste the URL, and write a few words about your content, then it is not because you lack these magical buttons.”

Sure, I get the the ‘magical buttons’ are not the be all and end all, but everyone knows that the consumer gets lazier and lazier everyday. If you don’t make it easy for them to do something then someone else will.

Siddharth Deswal
November 21, 2012

Laura,

Thanks for dropping by. A quick question: who do you want to talk to, the lazy consumers or the active, engaged ones?

Nick Cobb
November 22, 2012

@Siddharth,

I think everyone would prefer the active, engaged consumers, however, preventing the lazy ones from sharing your content/product potentially keeps their friends from discovering your company or product, and certainly they have some “engaged, active” friends.

As for the article, of course their results stunk. The implementation of those social badges is awful.

Juho Tunkelo
November 22, 2012

While I agree with the hypothesis of the test and understand the results, I don’t see how that debunks the concept of social proof at all.

If anything, it underlines how important it is to actually provide some proof instead of a few lukewarm likes.

Social media isn’t the same as social proof, after all…
http://en.wikipedia.org/wiki/Social_proof

Siddharth Deswal
November 22, 2012

@Nick,

For a blog that regularly has very few shares/likes, I have to agree with what Information Architects wrote.

In that initial phase when a blog is hardly known, I’d say they should focus on having actual discussions instead of going for “shares”.

As an example, see ContentVerve.com. Michael (the person running it) doesn’t have too many “social media shares” as such for his posts, but absolutely great discussions after most most of them (http://contentverve.com/test-your-gut-feeling-which-cta-won/).

Point being, better to focus on the talking with your visitors than trying to get clicks of affirmation from them.

Laura
November 22, 2012

@Siddharth

Exactly what Nick said. My thoughts exactly. I also think that today’s consumer is only going to get lazier, but having them take that first action may lead to future engagement. If they don’t have the chance to take that initial action, when are they ever going to engage with you. Those lazy visitors could very possibly become engaged and profitable customers – if given the chance.

@Juho

I have to say that when I started reading this article I was thinking the exact same as you. I don’t think the writer got to the point of ‘social proof’ quickly or clearly enough to justify the title. However, the very likes, retweets etc that you get are social proof in themselves. It is someone saying, hey I like this enough to let other people know that I like it, and believe that it is valuable enough to share.

Stephen Olson
November 24, 2012

The test ran for 3 weeks at which point three goals had reached full statistical conclusion showing big wins and many of the other goals were also showing positive trends.

I am not clear about the conclusions. Did any of the goals have to do with social media engagement? If so then the big wins would prove that social media is good for a web site as long as it is done effectively. Just like everything else in SMM.

Anne Stahl
November 24, 2012

Great discussion, all! A few responses from me: first off we are not preventing anyone from using Social Media – we are removing only one of the three social media ‘bars’. The concept here is a lot more subtle. We looked at the kind of users who is likely to read this kind of page: highly technical users reading a detailed technical article. In THIS situation, the social media bar at the top of the page harmed conversions (sale inquiries). Further tests are needed to determine whether this is due to the low like count or the fact that the social media bar was at the top of the content. We can’t say if it was a factor of users getting distracted, or loosing trust due to not giving enough weight to content, or if the low like counts decreased the websites’ authority.

It is correct to say that if anything, this test further affirms the working of ‘Social Proof’. I’m more referring to the myth of by simply adding social media integration all over a page one would automatically gain social proof. And I see now that I didn’t say this clearly enough.
The next step I’m planning, is to test how placement, style, number of likes, follows vs likes, etc… impact conversions. This is but one study of many that have been conducted and merely shows that we need to learn more. But pending these results it’s safe to say, that any social media integration should be carefully assessed and then tested!

Anne Stahl
November 24, 2012

Also, Stephen, we are not testing for Social Media engagement, we are testing website engagement and Sale Inquiries.

Unfortunately we have not yet come up with the technology to close the testing loop around social media… even though we can repeatedly see that social media bars harm conversions in a single user cycle, we cannot say if that user, who didn’t buy, did perhaps by sharing, bring 10 new users to the site, 2 of which bought – thus increasing conversions.e.g. using query strings attached to the share, but have not yet concluded this method nor tested it. Stay tuned!

Shane Sparks
December 5, 2012

Social Media and Social Proof are two MUCH different things. Debunking Social Proof would be a blow to psychological research.

Social proof, also known as informational social influence, is a psychological phenomenon where people assume the actions of others in an attempt to reflect correct behavior for a given situation. This effect is prominent in ambiguous social situations where people are unable to determine the appropriate mode of behavior, and is driven by the assumption that surrounding people possess more knowledge about the situation.

Social Media may or may not be relevant or interesting to your market.

Shane Sparks
December 5, 2012

This test possibly debunks whether Social Media has an value as an agent of Social Proof.

Anne Stahl
December 6, 2012

Shane, you are absolutely correct. Social Media is NOT the same as Social Proof. But the classic term ‘Social Proof’ has been used in association with Social Media, in that when pages or products have many ‘Likes’ it means that users are more inclined to also like the page. This is because these users feel that other users opinions are very valid and more reliable than product manufacturers opinion. It does thus not match the classic definition of social proof, but it’s very close.

In this article however, I’m not discussing the term as such and/or how it relates to Social Media. The point here is simply that when there are FEW likes, displaying these likes acts like a deterrent. This may or may not prove ‘Social Proof’ for website applications, but it does debunk the myth that many marketers propel that by just adding the likes, they will produce the effect of ‘Social Proof’. And in review my headline should have been more accurately ‘Debunking the myth of adding Social Media ‘Likes’ acts as ‘Social Proof” and stating that this CAN be the case, but only if certain conditions are met. Ie. marketers really ought to better understand what Social Proof is and how it functions in order to more accurately predict how using Social Media will influence their website visitors.

Thanks for pointing out the correct use of the term here!
Anne

Philip Mahler
December 10, 2012

We conducted a social proof experiment at SPAMfighter and actually had this test featured at WhichTestWon (http://whichtestwon.com/spamfighters-facebook-icon-test)

Although the results were not the same sitewide we are positive that social proof plays a big part and is definitely worth a test or two.

Anne Stahl
December 10, 2012

Philip, From the screenshot it looks like your site had close to 53,000 likes! That is the kind of number where you clearly have social proof. In our test, because each blog page counted likes, they were fragmented and ranged from 0 to 12 only. Thus the suspicion that they act like the opposite, or rather, the psychology behind it is probably the exact same, people do what many of their surrounding peers also do. If not many like this page, they won’t either. If on the other hand many DO, then it is like a big sign of approval.
In addition, I have long since wanted to test ‘Recommend’ vs ‘Like’ – as I suspect that this word change is significant esp for certain sites.
Thank you for sharing your test Philip.
Anne

Leave a comment
Required
Required - not published


6 + = seven

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com