Visual Website Optimizer

Suppose you want to increase newsletter signups and you (rightly) decided to use A/B split testing for the task. You have all the ammo ready with you:

• Well defined problem (increasing newsletter signups). Check.
• Tools at your disposal (Visual Website Optimizer). Check.
• Go ahead from your IT department (for adding tracking code). Check.
• Excitement and confidence to make a significant improvement. Check.

But you still miss one essential ingredient without which all your ammunition will turn out to be useless. That ingredient is your test plan. Hypothetically, you can test everything on your page (from footer copyright notice to your company logo). But that assumes you have infinite resources and unlimited patience.  When you are doing an A/B test, you want to maximize potential returns for the time invested in creating and running the test. Since a split test takes time (2-3 weeks) to show results, choosing the right element to test and creating good variations for it becomes the most important step.

A great way to have interesting ideas for your A/B split or multivariate test is to study and read existing case studies.  Case studies related to your problem (e.g. newsletter signup) and your industry (e.g. finance)  tell you what was tested, which variations were created and the level of success that was seen. Great case studies go one step further and reflect on what lessons can be learnt from the test.  All in all, pouring through multiple case studies is a guaranteed  way to get  numerous good ideas for your A/B test.

A quick note of warning: NEVER replicate the results of a case study without conducting testing on your website. What worked for them will not necessarily work for you; though what worked for them has a good chance for also working for you (but it is never guaranteed). So, best strategy is to learn from case studies, get ideas but implement your own A/B test based on the insights.

Finding relevant and interesting case studies is really hard. For your convenience, I have compiled some excellent resources on where to find case studies for A/B Split and Multivariate Tests:

• ABTests.com – a repository of case studies
• Anne Hall’s WhichTestWon – a weekly updated blog on A/B test results
• MarketingExperiments.com – subscription-based but has a lot of free articles
• Google – simply search for “a/b test” “case study” increase signups (or downloads, sales, etc.)

As you can see there are multiple sources for interesting case studies. But finding case studies relevant to your challenge and your industry is still difficult. To help you with that task, we have created an interface to search case studies by problem type and industry type. Really happy to introduce…

This is the first iteration towards the vision of having an easy to use interface for accessing resources on A/B split testing. The toughest part of an A/B test is to come up with a good test plan and beginners can easily get de-motivated if they don’t see positive results for their first test. A/B Ideafox hopes to provide interesting testing ideas which can potentially produce successful results quickly because those ideas have already been tried elsewhere. (Search results can be a little non-specific for now but we are working hard to have refine the result quality. Leave a comment if you have ideas on on refining A/B Ideafox).

Let me know your feedback on A/B Ideafox and other case study sources by leaving a comment here. Do you think case studies really help in getting interesting ideas? What other resources (videos, screencasts, tutorials, etc.) can help for taking A/B testing to mainstream?