The Next Generation of Visual Website Optimizer is launching on 29th April 2014 See What's Coming

Selling cookies online? Next-day shipping offer may increase sales by 41%

Posted in A/B Split Testing, Case Studies on August 24th, 2011

Imagine that you are selling cookies online and want to increase sales. What would you do next? One way to increase sales would be to increase traffic that you get but that costs a lot of money. Another way would be to work on optimizing landing pages, product pages and checkout pages. You may test your button colors, call to actions or layout. If you are adventurous, you may move beyond simple changes and start testing entirely different offers or value propositions. Who knows if $5 discount increases sales or probably next-day shipping is more important to convince a customer to gift a box of cookies?

The client: online cookie store

These were exactly the kind of questions that (our certified partner) Trinity Insight wanted to answer for their client SmileyCookie. SmileyCookie is a niche eCommerce store that sells cookies online. They allow you to gift customized cookies to anyone for any occasion (weddings, sports events, corporate events, etc.) What could be a better gift than a box of customized cookies? They worked with Trinity Insight, a consultancy, to help increase their sales and conversions. Trinity Insight used Visual Website Optimizer for A/B testing and then shared their excellent case study with us. (Key point to note is that no internal IT resources of SmileyCookie were used in the test. All of the following was done using Visual Website Optimizer interface).

A/B test challenge

SmileyCookie has a large header location which they use for seasonal or promotional offers. It is at a prominent place and hence catches attention of the visitor. Since an eCommerce store can position itself in various different ways, Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors. They wanted to understand which aspect about SmileyCookie’s service is more important for visitors. Following is a screenshot that shows the header where these value propositions were tested:

Variations tested

Trinity Insight tested five different value propositions and designed their respective creatives. These were five different value propositions:

  • Order Today -> Ships Next Business Day
  • Want to save $5 OFF your next purchase? SIGN UP NOW ->
  • $6.99 Ground Shipping For Your Entire Order
  • FREE SHIPPING on any order over $40
  • Cookies Made Fresh & Hand Iced For You!

Following screenshot depicts the creatives that were developed:

A/B test results: 41% increase in sales

While the test was running, the overall conversion rate for the site was 8.98% with an average order value was $29.85, yielding a $2.68/per visit value. Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average. Note that this increase was statistically significant at 95% confidence level.

Here’s what Trinity Insight said about the results:

Prior to testing, SmileyCookie could only make educated guesses as to which core value resonated with their customers and an opportunity to generate an additional $1.10/per visitor would have never been realized. By continuing to look at ways to make the site more engaging to their customers and testing these opportunities, SmileyCookie is guaranteed to create an experience that’s better today than it was yesterday as well as learn great lessons along the way about what their customers truly find value in while visiting SmileyCookie.com.

Why next-day shipping worked?

Cookies are perishable items and when you are ordering cookies as a gift or for special occasion, you want them sooner than later. Next day shipping increased sales because it tells the visitor that they don’t have to wait for long to get the cookies. There is no uncertainty and visitors know exactly when they are going to get the cookies. Since these cookies are ordered as a gift or for special occasions, customers may not be very price conscious and hence value proposition regarding discounts or low shipping rate didn’t work that well.

Of course, all this is obvious after we have A/B test results. And in fact, all the reasons proposed above may not be true. Interpreting A/B test results is always hard because if you know why a variation works better, you don’t need to do A/B testing in first place. You can simply implement that variation directly without A/B testing.

But, nevertheless, interpreting A/B test results are useful to see what would work in future or for other similar websites. So, if you sell other such perishable items online (say cakes, food, ice-cream, etc.) emphasizing on how fast your shipping is may increase your sales. Try A/B testing it!

Hope you liked this case study. If you have any comments or feedback, let us know!

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


Tags

4 Comments

[...] Selling cookies online? Next-day shipping offer may increase sales by 41% [...]

Ryan
October 24, 2011

Please more case studies ;) I’m testing your visualwebsiteoptimizer now and I must admit it’s amazing and very user friendly tool.

[...] There are hundreds of special offers/promotions you can run during the holidays. Of course, the big question is which one to actually run on your website? Well, why don’t you A/B test different offers? One of our customers found out that a next-day shipping offer outperformed a $5 discount offer or a free shipping offer, and ultimately increased sales by 41%. [...]

[...] Selling cookies online? Next-day shipping offer may increase sales by 41% Category: Header/Banner Icon Tests Who conducted the test: Trinity [...]

Leave a comment
Required
Required - not published


eight + 1 =

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com