The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

What works better: a video or an image slider? A/B test finds 30% increase in signups

Posted in A/B Split Testing, Case Studies on May 10th, 2012

Video always has interesting effects on conversion rate or sales on a website. In a lot of cases, video helps in explaining the service or the product, and hence increases sales. In other cases, it may actually reduce conversions (probably because it is not a good fit for that website, or that video is poorly made or presented). Previously, we had featured several interesting case studies about videos. One of them was about A/B testing a video on a landing page, and how it increased conversions by 46%. The other one was about testing ‘Watch a Video’ v/s ‘Get Instant Access’.

The only real way to know what a video will do for your website (or landing page) is to actually conduct an A/B test. And that’s what one of our customers Device Magic did on their homepage. They tested existing version of their homepage (which had a video) against a new version which had a jQuery based image slider instead.

A/B test (video v/s image slider)

Device Magic Mobile Forms allows organizations to quickly and easily build robust data collection systems. Using their API, developers can build rich integrations featuring push technologies e.g. for work orders, proof of delivery, maintenance reports, retail surveys etc. They wanted to see if a video or series of rotating jQuery slides would work best for driving people to signup, so they conducted a simple A/B test using Visual Website Optimizer.

Existing homepage (with a video)

Actually, they weren’t sure that the video was of sufficient quality and they suspected it was too technical. So, they thought maybe slides would present their offering more concisely.

Homepage variation (with an image slider)

Can you guess which version worked better?

Results and Lessons

They measured two goals: % conversion from home page to sign up page and lastly number of signups completed. Initially their control (with video) was beating variation (with image slider) and they couldn’t understand why. But then they let the test run a while longer until they had a lot more data – much to their surprise, the result inverted to variation outperforming control, which is what they had expected. And this result was statistically significant.

The image slider variation increased conversions from homepage to signup page by 35%, and the total increase in subsequent signups was 31%. So, it was indeed true that for Device Magic, image slider worked much better as compared to the video.

A key learning from this test would be to get enough data that one is absolutely sure of the result, and that one shouldn’t be too hasty because some detail may be skewing the result or leading one in the wrong direction. Patience in A/B testing is the key learning here. (We have an A/B test duration calculator online to tell you how long you should run a test before giving up on it).

If you have had similar (or contrasting) experience testing video v/s image sliders, do let us know in the comments below.

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

10 Comments
Mark@ Make Them Click
May 11, 2012

They really need to understand why the video was winning at the start.

Such reversals/crossovers of test results are common and too be expected, but it doesn’t necessarily mean the slider option was the best option.

Depending on your product/service, campaign or whatever, your best, hottest and usually most valuable prospects will convert early in the test.

Once they are taken out, what you are left with is testing the response of the crud, laggards’ late adopters low value customers.

So it’s critical that you figure out who you want to design for: The early hot prospects or the late low value ones. Usually 80% of the revenue will come from the hot prospects.

Koji Sato
May 11, 2012

Great post!

Our customers have been quite successful with sliders, however as you state, testing is the key since this should never be assumed.

Considering the usability aspect, for me, I find if a taboo to auto-start a video, and from the test in this post, it looks like it was to be triggered on click. Another great thing about a slider is that the standard is for them to play automatically on page load, thus stepping the viewer through, with no specific interaction required.

ash
May 11, 2012

i think its less about the content (vid vs image). the first screen is cluttered with tabs and bullet points and the vid is off to the left. the second screen is clean, front, and center – its easy to digest

Cooper M
May 13, 2012

You’re right on the money when you said ‘probably because it’s not a good fit for the website’. It’s all about which one expresses the value of the specific business best. Either way, I appreciate the post because I’ve been biased towards videos lately..

I think the mixture of both in content of your website works better and its up to you how you want to present your product but I believe if you want to use video than it should be properly created and reader can easily understand your products.

So, I believe using images or video its up to the author how you want to use it but use it in properly.

Graham E
May 18, 2012

There’s a big difference in the headlines and the way the eye is drawn to the content as ash suggests, I think this may have contributed to the win – it’d be interesting to see a follow up on a multivariate test including different elements of each in the design.

Idria
May 24, 2012

I have to agree with Graham, I would present this as a range of multi-variates rather than a simple split. Too much has changed on the page to define it by a single albeit large element.
Also know what it’s like when a client has a “video” that MUST be put on their site even though it is neither irrelevant to their core business nor engaging for visitors. Finally some testing ideas to present to them!

Jaime
June 1, 2012

My friends from South Africa are terrible at videos. It does not surprise me to see that image sliders was better than that boring 5 minute demo with Dusan’s voice which sounds like a lullaby.

David
October 23, 2013

If you have a highly engaging video that is well placed, and easily shareable it should beat any other form of marketing hands down, take Dove’s Real Beauty Sketches Video http://www.youtube.com/watch?v=XpaOjMXyJGk&feature=share&list=PL0BRaXBPJ6iZMJoRM9TFRc2Mq4d0KmKqZ Running at over 56 million views to date..

This may not be a quick how to video to direct users around your site but it does show the power of Video when done well.. over 56 million views world wide since April 2013 is pretty impressive and does show the power of video done right!

A poor video would also skew the results one would imagine?

Siddharth Deswal @ Wingify
October 24, 2013

David,

A poor video would almost certainly skew the results. I think it’s finally about communicating whatever it is one is trying to say.

Video, with the inherent animation and sound is certainly better than a bunch of images on a slider. Also, sliders have shown to decrease profit as most people need more than 3 or 4 seconds to really think about what’s being shown in the sliders.

And then finally, there are folks who combine images, videos and ‘social’ to create something very interesting, like Lexus at https://www.youtube.com/watch?v=DgmQV7hGQXM

Leave a comment
Required
Required - not published


− 5 = two

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com