Imagine that you are selling cookies online and want to increase sales. What would you do next? One way to increase sales would be to increase traffic that you get but that costs a lot of money. Another way would be to work on optimizing landing pages, product pages and checkout pages. You may test your button colors, call to actions or layout. If you are adventurous, you may move beyond simple changes and start testing entirely different offers or value propositions. Who knows if $5 discount increases sales or probably next-day shipping is more important to convince a customer to gift a box of cookies?
These were exactly the kind of questions that (our certified partner) Trinity Insight wanted to answer for their client SmileyCookie. SmileyCookie is a niche eCommerce store that sells cookies online. They allow you to gift customized cookies to anyone for any occasion (weddings, sports events, corporate events, etc.) What could be a better gift than a box of customized cookies? They worked with Trinity Insight, a consultancy, to help increase their sales and conversions. Trinity Insight used Visual Website Optimizer for A/B testing and then shared their excellent case study with us. (Key point to note is that no internal IT resources of SmileyCookie were used in the test. All of the following was done using Visual Website Optimizer interface).
SmileyCookie has a large header location which they use for seasonal or promotional offers. It is at a prominent place and hence catches attention of the visitor. Since an eCommerce store can position itself in various different ways, Trinity Insight wanted to test which value proposition works best for SmileyCookie visitors. They wanted to understand which aspect about SmileyCookie’s service is more important for visitors. Following is a screenshot that shows the header where these value propositions were tested:
Trinity Insight tested five different value propositions and designed their respective creatives. These were five different value propositions:
Following screenshot depicts the creatives that were developed:
While the test was running, the overall conversion rate for the site was 8.98% with an average order value was $29.85, yielding a $2.68/per visit value. Over this same period of time, the winning creative, Order Today -> Ships Next Business Day, had a 12.61% conversion rate and a $29.95 average order value resulting in a $3.78/per visit value, a 41% increase over the site average. Note that this increase was statistically significant at 95% confidence level.
Here’s what Trinity Insight said about the results:
Prior to testing, SmileyCookie could only make educated guesses as to which core value resonated with their customers and an opportunity to generate an additional $1.10/per visitor would have never been realized. By continuing to look at ways to make the site more engaging to their customers and testing these opportunities, SmileyCookie is guaranteed to create an experience that’s better today than it was yesterday as well as learn great lessons along the way about what their customers truly find value in while visiting SmileyCookie.com.
Cookies are perishable items and when you are ordering cookies as a gift or for special occasion, you want them sooner than later. Next day shipping increased sales because it tells the visitor that they don’t have to wait for long to get the cookies. There is no uncertainty and visitors know exactly when they are going to get the cookies. Since these cookies are ordered as a gift or for special occasions, customers may not be very price conscious and hence value proposition regarding discounts or low shipping rate didn’t work that well.
Of course, all this is obvious after we have A/B test results. And in fact, all the reasons proposed above may not be true. Interpreting A/B test results is always hard because if you know why a variation works better, you don’t need to do A/B testing in first place. You can simply implement that variation directly without A/B testing.
But, nevertheless, interpreting A/B test results are useful to see what would work in future or for other similar websites. So, if you sell other such perishable items online (say cakes, food, ice-cream, etc.) emphasizing on how fast your shipping is may increase your sales. Try A/B testing it!
Hope you liked this case study. If you have any comments or feedback, let us know!
Previously on this blog, we talked about impact of A/B testing on SEO. Now, in this post we talk about how to calculate ROI for your SEO efforts. This is a guest post by Prashant Puri, who is co-founder of AdLift.com – a niche SEO firm in the SF bay area. Previously he was Head of Global Search for Shopping.com (an eBay Inc Company). You can follow/reach out to him on Twitter – @puriprashant
“Measurement is the first step that leads to control and eventually to improvement. If you can’t measure something, you can’t understand it. If you can’t understand it, you can’t control it. If you can’t control it, you can’t improve it.”
- Anonymous
The sentences above basically sum up the importance of measuring your SEO initiatives – only if you can measure SEO ROI can you improve it. Here’s how you get started with it:
You must exclude your own branded keywords from analysis before calculating ROI. This needs to be done because many users will search your brand on search engines instead of typing in your URL directly. But those searches aren’t due to your SEO efforts, hence they need to be excluded. Use your analytics tool to segment keywords into 2 buckets:
Here’s how you can do this (using Google analytics as an example) –
Once you exclude your brand terms – plot the traffic graph for your non-brand terms.
It’s easy to export this data and calculate the % increase in traffic or revenue (if you have ecommerce tracking in place – highly recommended). Once you have the revenue number you can calculate the ROI from your SEO efforts –
It’s relatively easy for ecommerce websites to calculate this once you have the tracking in place; however, the tricky part is for non-ecommerce websites to estimate ROI on SEO. This is where tracking conversions and setting goals becomes crucial – a conversion might be a sign-up, email opt-in, of just pageviews which drive up your advertising revenue.
Once you’ve identified your conversions and set this up the next step is to attribute a revenue number with each conversion. Let’s say you make $5 RPC (revenue per conversion) – so if you drive 50,000 incremental visits at an average conversion rate of 5% and a RPC of $5 – your revenue generated is 50,000 x 5% x $5 = $12,500. Now that you have the revenue number you can calculate your SEO ROI.
SEO helps in increasing the incremental visits as you continuously invest in content and link development strategies. Another lever that you can pull to maximize revenue is the conversion rate on your website. Visual Website Optimizer is a great tool that can help you A/B test landing pages to improve overall conversion rate.
Other SEO key performing metrics that help you analyze the performance of your SEO efforts are –
1) Number of Keywords driving traffic: This is a great metric to track your overall on-page optimization efforts. This tells you the number of keywords driving traffic to your site. In Google analytics go to Traffic Sources -> Search Engines -> “non-paid” -> under source chooses “keywords”.
2) Number of Landing Pages: Another great metric to track, this tells you the number of unique landing pages that are driving traffic to your site. As your SEO traffic start to grow you should start to see this number increase.
Search Engine Optimization has evolved in to being a highly effective marketing channel – and its exciting to be able to track this very closely!
For eCommerce websites, having the right product mix is very important. If you don’t have products that your customers want, they are going to leave the website and never come back again. And unless you are Amazon or Walmart, you cannot afford to have all sorts of of products to offer (handling inventory, shipping, policies is a logistical challenge so most eCommerce companies specialize in specific niches). So knowing which products to offer is a big challenge for online retailers. This problem becomes even more difficult for the fast-changing technology products such as cell phones.
One of our customers, Mobal, sell international cell phones online. Since the primary product they offer is a cell phone number that works across the globe, choice of handset is a secondary factor in a purchase decision. Or, so they thought! When they A/B tested a new line of handsets by offering additional Android option, they saw sales grow by 27%. This significant increase happened just by adding one new product! No design changes, no new cell phones plans — just a new handset option.
Their goal was to determine whether replacing their current deluxe phone with an Android smartphone, or adding an Android smartphone to their current offering would provide an increase in sales over the current 3 phone models they offered. They ran an A/B test on their international cell phones landing page which gets traffic from organic, PPC and direct sources.
Control Page – Budget, Classic and Deluxe Phone
The primary motivation for this A/B test was to see whether there is any demand for smart-phones from international travelers and also to assess the impact of offering a smartphone on their other products. Since international travelers value service and network coverage, they were unsure which variation would convert the best.
They started a test with 3 pages just changing the phone product section of each. One was the original page (control) that you saw above. Other variation looked like below screenshot:
3 options variation (with Android option). 13% increase in sales. 
The results were very surprising with the 4 phone option (see a screenshot below) converting 27.77% better than the control page, they felt that an additional phone option may have had an adverse effect on the overall conversion rate but the test proved them wrong. It was also interesting to see that replacing their current deluxe phone with an Android handset saw an increase in conversion over the control page of 13.3%. Clearly, Android phones worked like magic and helped them gain a lot more customers!
4 options variation (with both Android and Deluxe phone option). 27% increase in sales
From this A/B test, they concluded that offering four phone options meant a significant increase in the number of phone sales we had and that there was a demand in the market for smartphones.
Since they used Visual Website Optimizer for their split test, they gave us a testimonial:
Visual Website Optimizer allowed us to simply and efficiently conduct a split URL test for our 3 options. The easy to use dashboard system meant that we could identify which product selection offered us the biggest increase in sales.
So, if you are an eCommerce website, make sure you regularly A/B test your product mix. You never know which new products may increase your overall conversions and sales!
This is a guest post written by Felipe Wesbonk who is a consultant with Traffic Builders, one of the Visual Website Optimizer certified agencies. They extensively use integration of Visual Website Optimizer with Google Analytics and in this article, Felipe talks about how this integration provides them with many additional insights.
Google Analytics provides many more insights into the performance of your website and online marketing campaigns when using plugins of various useful tools, like Visual Website Optimizer (VWO) and BTBuckets. Also by using the Google API it’s possible to make custom dashboards, in which external data can be integrated. During the GAUC (Google Analytics User Convention) in Amsterdam last month, I gave a presentation about these plugins. In this article I explain the integration from VWO in Google Analytics and the way you can use this plugin in making custom reports.
It is very easy to get the most out of Visual Website Optimizer within Google Analytics. Here is the plugin:
if(typeof(_vis_opt_GA_track) == “function”) { _vis_opt_GA_track(); }
Just put this in the GA snippet and make sure you put this above the _gaq.push(['_trackPageview']);. That’s all you have to do. Now the fun part begins.
With the special Google Analytics plugin of the Visual Website Optimizer you are able to integrate the data from VWO with Google Analytics and the e-commerce module. The plugin puts the data of the various variations on custom variables back into Google Analytics. As default, you will see the VWO data in slot 4 of the custom variable.
When the data is here, all kinds of customized reports can be created. This way you can add some metrics, which you do not see in the standard reports in Visual Website Optimizer. You are able to make custom reports and the analysis in order to decide if the variations give the conversions which are satisfying in terms of reach and engagement. This will get you one step closer to an integrated optimization strategy.
As you can see in the screenshot above, it is quite easy to make a custom report with the metrics you find useful. In the filter you can use a regular expression to make sure you get the right data from the custom variable slot 4.
Using custom reports you are able to see which channels drive traffic to the different variations. Choose the most valuable goals as the metrics. In the Dimension Drilldowns you select ‘custom variable (Value 4)’ as first dimension and ‘medium’ as second dimension. Now let’s take a look at the winning variation, which you put into the filter of the report. You can use the custom report to see which channel is still behind in conversions.
The example below shows that the medium ‘referral’ and ‘affiliate’ don’t convert well. Besides, they also have a very high bouncerate (46% and 80%). In this case I would recommend to take a close look to see which sites (sources) from ‘referral’ and ‘affiliate’ give the most traffic. On these sites, you should change the banners and links to make sure it will fit better with the best performing variation in order to achieve a higher CTR and more conversions from those sites.
This is one of the custom reports you could create to get more valuable insights for the best performing variation, in order to make a more informed decision. If you have any suggestions for custom reports which you can make with the data of the VWO tests, please let us know in the comments!
