The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

Facebook Login Reduces Ecommerce Sales (Case Study)

Posted in Case Studies on August 22nd, 2013

It is generally accepted that an easy ecommerce checkout experience will result in increased sales and better conversion rates. One of the best ways to do that is to provide Facebook login during the the checkout process. After all, research from Social Labs shows that almost 50% of ecommerce visitors are logged in to Facebook and it takes just two clicks to authorize a website to collect your name and email.

That’s bound to increase conversions, right? Not exactly.

BliVakker

BliVakker.no is one of Norway’s leading online cosmetics retailer with about 20,000 visits per day. They’ve been working on optimizing the checkout process for the past 6 months. The improvements were based on findings from usability tests, best practices from leading ecommerce sites, A/B tests and their own analytics.

The “Facebook Login on Checkout” Test

Blivakker used Visual Website Optimizer to run a split test on 8000 visitors where half saw a login page with an email field and a Facebook login button, and the other half saw only the email field.

Control: Checkout process with Facebook loginBlivakker login page with FB Connect

Variation: Checkout process without Facebook loginBlivakker login page with FB Connect

Why did they run this test?

The most interesting reason ever: a developer pointed out that the Facebook login was adding significant complexity to their internal systems and processes, so was it really worth it? That question spurred them to research Facebook login best practices as well as how they’re used in other ecommerce stores.

The result: 3% increase in conversions and $10,000 in revenue (in just seven days)

The login page without the Facebook Connect increased conversions by 3%, which at Blivakker’s scale translates to about $10,000 in extra sales per week.

The important bit

Also, get your own damn A/B test

Just like the image above, you need to Get Your Own Damn A/B Test. Best practices are great when you’re initially creating a checkout process, but never let others’ data dictate how you setup your ecommerce store. Ensure that you’ve started your tests before any paid campaigns so that you can stop losing sales and revenue as soon as possible.

Here’s the comparison image. If you’ve got friends who run Ecommerce stores, you really should share this with them.

BliVakker Ecommerce Checkout A/B Test

Image Credit
Product photo of the awesome “Get your own lighter” by Valiantstudios

Siddharth Deswal

I do marketing at Visual Website Optimizer.

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

14 Comments
B. McKenzie
August 22, 2013

Is this actually statistically significant? A lift of 3% would only be statistically significant (at 95% confidence) with a sample size of 4,000 & 4,000 if the control group’s conversion rate were at least ~60%. For 99% confidence, which I think should be the goal here, the control group’s conversion would had to have been at least ~73%.

If the level of confidence is less than 99%, I’d recommend to keep the test running at least an additional week. I’d also recommend checking whether there is significant seasonal variation in how often your visitors had used Facebook to log in. If you notice seasonal trends, I would recommend keeping this test running and segmenting the results by month.

Gregory
August 22, 2013

Facebook Login option is still available on their site :)

Siddharth Deswal @ Wingify
August 22, 2013

Hi Brian,

Yes, the result was statistically significant at 99%. The control group’s conversion was more than 75%.

About seasonal trends, I’m not sure what kinds could happen here. Essentially, why would buyers login using FB at certain times of the year and not otherwise, unless of course it was tied in to a discount of some kind?

What do you think?

B. McKenzie
August 22, 2013

“Essentially, why would buyers login using FB at certain times of the year and not otherwise, unless of course it was tied in to a discount of some kind?”

A few possibilities came to mind:
–A promotion or discount, particularly anything they run on Facebook.

–Seasonal interest in different products. For example, in the U.S., there are frequently demographic differences between holiday shoppers and non-holiday shoppers… In the nonholidays, I speculate that a larger proportion of their customers would be buying for *themselves*, whereas there will probably be an increased proportion of gift-buyers in the holiday season. Personally, if I were buying cosmetics for a girlfriend, I’d be more likely to try a Facebook login than create an account for a website that I would only use for Christmas and perhaps Valentine’s.

–Another possibility here would be that there may be seasonal variations in average age of user. For example, I would guess that the average age of U.S. flower buyers drops significantly in May because that’s when most high schools have prom — perhaps there is some such fluctuation for cosmetics? If so, I would guess that younger people are probably more receptive to Facebook…

–Any significant changes in traffic sources (e.g. an expansion or contraction of PPC advertising at certain points of the year).

By the way, did this test run for both new and returning visitors? If so, were there any significant differences in how the groups fared? It looks like the alternative to Facebook was logging into an existing account or perhaps creating an account (I’m not sure — I don’t speak the language). A returning customer might be more receptive to using his/her existing account on the website, whereas a new customer probably wouldn’t have one already.

Profit Inzerce
August 23, 2013

Bad news… But I still can see the FB login on their site.

Siddharth Deswal @ Wingify
August 23, 2013

Brian,

Not all trends you mention are “seasonal”. For example – the FB promotion or PPC campaign. These are conscious decisions by the business to drive a certain kind of traffic to the website.

But I completely agree with the bit about certain kinds of visitors flocking to sites before prom or Valentine’s Day or even graduation, and these buyers being more likely to use FB Connect. So thanks for these points, one learns something new everyday :)

About the test running on new or old visitors, I don’t have that data. I’ve asked Karl from BliVakker to jump in the discussion, so hopefully he should provide more insights.

Shrey Sehgal
August 23, 2013

Hi Siddarth,

Here you have taken an assumption that Facebook login is used to for better conversion rates and fasten the checkout process.

The numbers will work with this assumption.
But there is much more to facebook login than just a quick checkout process. The data acquired through facebook login could be catalyst to repeat purchase behaviour if used wisely. Moreover it can be used to build engagement with the Customer.

So the site manager should ideally compare the drop in checkout process with ROI of the data and take decision to keep Facebook Login or not.

This could be the reason that they are still running Facebook login even after the fall in conversions rate.

Siddharth Deswal @ Wingify
August 23, 2013

Hi Shrey,

Thanks for dropping by. There are no assumptions in this case study, it’s all data collected in Visual Website Optimizer and information provided by the person who ran the tests.

About the reason why they still have the FB login is because they are not completely convinced that it is bad. Quite likely that it holds true only for certain kinds of visitors that, as Brian said above, are seasonal in nature.

Other than that, have you tried FB login in FlowerAura? Has it worked for you?

Etienne Juneau
August 23, 2013

Great test!

Clear results: I don’t see why other commenters argue.

I’ll take the extra $10,000 a week any day.

-Etienne

Dev
August 23, 2013

This is a pretty good AB testing case. However, it will be more interesting to know the exact drop-off point. If the users are dropping off in the “middle of facebook autentication” Vs “on successive pages” Vs “at payment gateway” will be more helpful in taking a definitive action.
Not everything can be reasoned out in an AB testing, but a 3% change is significant and should be reasoned out to take an apt action.

KP
August 23, 2013

I ran this test over the summer. A developer brought up the question and I wanted to do a quick test to have something more to discuss. The speed of doing a test is one of the great things with VWO.

We haven´t had a chance to discuss internally and we think it would be quite harmful to remove the FB login for the people that actually use it. One option would to make it less prominent in the user interface.

Patrick Curl
August 23, 2013

Great post, but — it’s also not an english site, so user trends in the country of origin could be off from what you might find in the UK, or America — so yes, it is definitely important to do your own split testing to see which works better– BUT I also agree w/ what someone said about how you use Facebook.

If you make 3% less conversions on the FB logins, but those logged in by facebook share stuff on their timeline, then push 15% more traffic to your site, then you’re still going to experience more revenue as a result, even though 3% less of that 15% boost may purchase.

Tomáš Kapler
August 24, 2013

One recommendation from a pro – show fb login to customers comming from fb.

And otherwise – this test is bit stupid, the problem of the form is that there is only one registration – fb. the e-mail part do not look like a registration. So people who do not want to login or login using fb simply go out. The a/b test just shows, that THIS version of form is bad, not that facebook login reduces sales.

Jan Horna
August 25, 2013

Totally agree with Tomas Kapler.

- The headline is very misleading. This test is not about the negative impact of Facebook Login. In this test you compare a single-option registration page versus a more complex, 2-option registration page.

- You could replace Facebook Login with the Twitter or G+ logins and get similar results (just my assumption).

- Regarding the FB Login part, why do you duplicate the headline “Logg inn med Facebook” on the button itself? I would go with just a button and some explanation below the button. See our site http://www.concertin.com. We ran many experiments with the button, button text, background etc. Little details can decrease or increase the FB Login conversion significantly.

Leave a comment
Required
Required - not published


× 5 = twenty

Notify me of followup comments via e-mail.

RSS feed for comments on this post.

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com