The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

Web designers and developers, we have changed A/B testing forever

Posted in How To, News on March 29th, 2010

Edit: The VWO Editor has undergone a complete makeover since this article was written. For updated usage of the features mentioned here please see the following articles in our knowledge base:

Warning: if the words CSS, JavaScript or jQuery scare you, it’s best not to read this alone. Rather, you should take help from your designer friend to help you understand how Visual Website Optimizer has become the most flexible A/B and MVT testing platform ever. Don’t worry: you still have the world’s easiest WYSIWYG interface for creating A/B and MVT tests. It is just that we have added new functionality to make VWO geek-friendly.

Introducing Advanced Code mode

Visual Website Optimizer has a great, full featured WYSIWYG editor (thanks to TinyMCE) for creating variations. So now you can edit your page section by clicking on them and select Edit operation from the operation menu. Take a look at the WYSIWYG editor below:

It was designed so that users can be saved from all the hassles of editing HTML and uploading images on their servers (WYSIWYG editor has integrated image uploader). However, some of our web designer users gave feedback that the WYSIWYG editor is limiting for them and they prefer swimming in HTML rather than avoiding it. So, what we have now provided an option of using a non-WYSIWYG editor (based on CodeMirror, thanks Eric for the suggestion). Any element’s HTML can be edited by selecting that element and choosing Edit HTML operation from the operation menu. Take a look how non-WYSIWYG editor looks like (for the same variation as above):

Like all CodeMirror awesomeness, it offers syntax highlighting (for HTML, styling and JavaScript), tab intendention, bracket pairing and unlimited undo-redo. Initial testers of this feature say that CodeMirror dramatically improves productivity as far as creation of A/B and Multivariate tests is concerned.

JavaScript API and beauty of JQuery

Advanced mode also gives you an API using which you can dynamically modify your variation before it is displayed on the page. Do you see “Insert JS API” button in the screenshot above? Clicking on it inserts the following JavaScript function in your variation:

<script type='text/javascript'>
// The following API function is called just before a variation is inserted.
// The variation is passed to the function as a (wrapped up) DOM element.
// Control is a DOM element on the page which will be replaced by the variation.

_vwo_api_section_callback[VWO_SECTION_ID] = function(variation, control)
{
  // jQuery is available as shorthand vwo_$. Example: vwo_$(variation).css('font-size', 'larger');

  // INSERT YOUR CODE HERE

}
</script>

As it is obvious from the code above, _vwo_api_section_callback[VWO_SECTION_ID] is a function that is called and the variation and control are passed as DOM elements. This allows you to modify the variation dynamically using your own custom JavaScript code and existing website content (control). Consider following use cases:

1. You want to test if including search keywords in the heading increases conversion rate. Here is how you will define a variation:

<p>Welcome to MySite, you searched for <span class="keywords">keywords</span>
on <span class="search">search engine</span></p>

<script type="text/javascript">
_vwo_api_section_callback[VWO_SECTION_ID] = function(variation, control)
{
  vwo_$(variation).find(".keywords").html(yourFunctionForKeywords());
  vwo_$(variation).find(".search").html(yourFunctionForSearchEngine());
}
</script>

2. You have an eCommerce site with multiple products and you want to test the phrase “Buy this for $45″ v/s “Buy now for $45″. However the pricing is variable as it is dependent on the product. So, what you will do is to extract pricing from the control and inject it into the variation.

Control can be for a single product while designing. While the test is running, control will be whatever is present on the (product specific) page:

Control:

<div>Buy this for $45</div>

Variation to extract price and use it:

<div>Buy now for <span class="price">$ something</span></div>

<script type="text/javascript">
_vwo_api_section_callback[VWO_SECTION_ID] = function(variation, control)
{
  var phrase = vwo_$(control).find("div").html();
  var regEx = /^Buy now for (.*)$/g;
  var match = phrase.match(regEx);  
  var price = match[1];
  vwo_$(variation).find(".price").html(price);
}
</script>

Lots of other examples can be given: greeting returning visitors, ad campaign specific message, testing latest comments or twitter box, etc. In a nutshell, where ever you need to test dynamic content you will find this API useful.

Limits are endless: the API gives you all freedom of server-side testing (where you generate different pages programmatically) with all the benefits of client-side testing (no page tagging, faster creation of tests, no involvement of IT resources, and best of all not touching site code at all).

Specifying CSS path of the sections in a Multivariate test

Imagine you have a page where the same element (say a button) is repeated at multiple different locations and you want to do an A/B test while making sure all buttons get the same variation. For example, take a look at the following page:

What if you want to test all “Buy” buttons at once (and that too without page tagging)? Also, remember featured products can change all the time so you can’t simply test the whole section. Here is how you will tackle the problem. Simply select one of the buttons and perform the required operation. Then you can change the CSS path of the created section to a generic one, like the following:

During the test, all elements matching this CSS expression will be replaced by the same variation. Using the same method (by giving a generic CSS expression) you can also present a consistent experience if elements being tested are on different website pages and at different locations.

Testing of style sheets and JavaScript: decoupling website content

Using the same advanced code mode, you can manually define a section which has no HTML content, therefore isn’t linked to any specific website content. Rather this type of section contains style, script or link tags that are injected in a page’s HEAD section. This way you can test different style sheets or themes (defined inline or linked externally) across your website. Or you can use it to have different visitor experiences triggered by variation-specific JavaScript or JQuery code. You create a section by simply clicking on any element.

Once you create a HEAD section, you can define variations using the same CodeMirror editor:

To summarize, advanced code provides following new functionality:

  • Syntax highlighted editing of variations
  • JavaScript hook with variation and control as the arguments
  • Inbuilt JQuery to be used in API
  • Defining a section’s CSS path
  • Testing of stylesheets or JavaScript code independent of any site content

The flexibility that Advanced Code mode provides for A/B and MVT testing is simply overwhelming. This mode enables all kinds of testing and should be a web designer or developer’s delight. We worked hard on this feature and would love to hear feedback from the designer and developer community. Do you like the new feature? What would you like to add to it?

 

Paras Chopra

CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!

The Complete A/B Testing Guide

Know all that you need to get started:

  • What is A/B Testing?
  • Is it compatible with SEO?
  • How to start your first A/B test?
Show Me The Guide


18 Comments
Steffen Binas
March 29, 2010

Nice…I just started with jQuery…it’s good to see it in VWO. So this nice AJAX-thing would be possible?

vwo_$(‘#myid’).load(‘newcontent.html’);

But I assume the content would be displayed delayed this way…give it another try:

jQuery.ajax({
url: ‘newcontent.html’,
success: function(result) {
vwo_$(‘#myid’).html(result)
},
async: false
});

Paras Chopra
March 29, 2010

@Steffen: Yes, that should be possible. Try it out :)

[...] split test on dynamic content? NOTE: This feature has been deprecated in favor of new, simpler JavaScript API which won’t require any tagging. Details [...]

[...] maybe you are testing different sidebar location or testing different placement of ads. While with CSS testing this is anyway possible without the hassles of creating separate versions of your complete website, [...]

avery
May 3, 2010

stuff like this that makes the internet so damn good

Ophir Prusak
October 7, 2010

Just a short note on example #2 “test the phrase ‘Buy this for $45′ v/s ‘Buy now for $45′”.

It would be much simpler to just test the static word “this” vs “now” and leave the rest of the content alone :)

Jens Schriver
October 15, 2010

I’m trying to run a test which has a variable, similar to your “Buy this for $45/Buy now for $45″ test above.

I have this section on about 100 tag pages:
Reports Tagged: wrestling​

The only thing that varies is the “wrestling” word. I’d like to test a variation with different copy, such as:
Reports Tagged: wrestling – scroll down to learn more.​
… where “wrestling” is replaced by whichever word is on the respective tag page.

I’ve tried to use your example above, but with no luck. Any pointers?

/Jens

Jens Schriver
October 15, 2010

Ups, no HTML code.

I’ve added the code here: http://pastie.org/1222138

Eugene
January 24, 2011

Guys the phraze above sounds wrong:
“Limits are endless”, it means the opposite to “there are no limits” what you probably tried to say. Thx

[...] Advanced A/B testing [...]

[...] get bored (and frustrated) if we think it has been long since we last stretched the limits of A/B testing. So, in order to kill boredom and feel happy, we added a new feature to Visual Website Optimizer. [...]

Bogdan Gribincea
June 1, 2011

How exactly does one create an A/B test (with multiple variations) using a Multivariate test?

Paras Chopra
June 1, 2011

@Bogdan: I’m sorry I didn’t understand the question. Can you explain?

Bogdan Gribincea
June 1, 2011

Hi Paras,

The note at the top of the articles says “So if you want to use it, we recommend you to use multivariate testing option (it also allows you to create A/B test).”

I want to know how exactly do I create multiple A/B variations in Multivariate mode. I can only figure out how to make it happen when changing just one element/section but not when I want to make multiple changes per variation.

Thanks.

Paras Chopra
June 1, 2011

@Bogdan: if you want to change multiple elements on a page, I’d recommend not using multivariate testing option and directly use A/B testing option in Visual Website Optimizer.

Bogdan Gribincea
June 1, 2011

That makes perfect sense but I thought I was missing something.

The point is that I’d like to have the new advanced mode in A/B too. Multivariate tests aren’t viable for pages with a small number of visitors but the ability to use CSS selectors to target changes would be useful for those cases as well.

Paras Chopra
June 1, 2011

@Bogdan: yes, that’s a great idea. Will add it to our development roadmap.

[...] dynamic product page. Here’s an example. (Editor’s note: they used Visual Website Optimizer’s advanced mode to create a test that runs across thousands of dynamic product pages on [...]

Leave a comment
Required
Required - not published


8 + = sixteen

Notify me of followup comments via e-mail.

RSS feed for comments on this post. TrackBack URL

I ♥ Split Testing Blog


Stay up to date

Recent Posts

Write for this blog!

We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com