The Next Generation of Visual Website Optimizer is launching in April 2014 See What's Coming

About Visual Website Optimizer

Possible Concerns

General Questions

How does Visual Website Optimizer work?

  • When you create your first A/B test, you get a one-time code snippet from Visual Website Optimizer (VWO). Add this code to your website’s HTML and you’re good to go. Create unlimited tests anywhere on your website. There’s no need for you to add the code again.

  • When visitors land on the page on which you’re running a test, VWO code calls for the test pages set up by you during the design phase. These pages are then randomly served to visitors who become a part of your test(s).

  • If it is a split URL test, VWO divides the traffic between the two (or more) URLs that you’ve set up for your test. Visitors are randomly directed to these URLs.

  • Once the page is displayed and the conversion goal is completed by the visitor, Visual Website Optimizer logs it in your conversion data.

Technical Explanation:

As soon as a visitor lands on your website, the JavaScript code snippet inserted by you in your website’s HTML contacts the VWO server.

Two parts of the response is then sent by the server, which comprises of:

  1. Static JavaScript code for running the test.
  2. Dynamic test data, which has specifically been set by you for the test that is running on the current page URL.

Both the static and dynamic responses are heavily cached by us at our backend. This keeps response times minimal. The (g)zipping and minification of the code reduces these response times even further.

On the basis of the server response, the elements in the DOM (Document Object Model) are automatically replaced to display the variation that the visitor has been assigned.

In case of a Click, Form submission or Engagement as a conversion goal, a VWO function binds to a click event on the page that checks if a conversion has occurred.

After section replacements and binding of goals, the page is displayed to the visitor.

How much improvement in conversions can you expect?

Conversion improvements depend on a lot of factors.

Running small A/B tests, like button color tests or headline tests usually have far lesser impact than running tests with big changes.

Often when your test hypotheses are backed by background research like usability findings, heatmap analysis, or customer feedback, you can expect higher improvements in your website conversion rate.

A/B test results can still be quite unpredictable and best practices may not always work. It is recommended that you test any changes before you make them permanent on your website.

Typically, you should be happy with at least >10% improvements in conversions. In some cases, you can even celebrate a conversion lift of more than 300%.

Read existing VWO case studies to get a balanced perspective.

What types of conversion goals are supported by VWO?

  • Click on a link - Specify a URL when setting up a test. The moment it is clicked on your test page, it will be counted as a conversion goal. For example, if your conversion goal is the download event, you will need to specify click on the download URL (the value contained in href attribute of a tag) as the conversion goal. And when a visitor completes this action, a conversion goal will be registered by VWO.

  • Visit to a page - Submit the URL in your test for which the goal needs to be triggered. If a visitor arrives on this URL from your test page, a conversion goal is logged into the data. For example, if you want to track visits to your features page from the homepage, the conversion goal will be triggered in VWO as soon as the visitor is redirected to the features page.

  • Form submission - This conversion goal triggers if the specified form is submitted by the visitor.

  • Engagement - It records clicks on any link (i.e. anchor element), button, or input element (with ‘type’ attribute equal to image, submit or button) present on the page. Check out this link for more details.

  • Custom Conversion Goal - This option is for technically-advanced users who want more than the default goals available in Visual Website Optimizer. You will get a JavaScript code, a call to which will trigger the conversion goal.

  • Revenue Goal - This option enables you to track revenue metrics, like total sales, revenue per visitor and average order value.

  • Click on Element - This goal tracks a click on any DOM element (or collection of elements) on the page. See this link for more details.

Miscellaneous Information About the Tool

  • Full factorial approach to multivariate testing

  • Uses cookies to track visitors

  • Visitors see same variation even on repeat visits

  • Conversions can be triggered even on repeat visits

  • The same test can run on multiple pages on the website

  • Multiple tests can be run on the same page

  • The test results are realtime; there is no lag

  • To understand statistical concepts that form the basis of Visual Website Optimizer, read this blog post

Where is Visual Website Optimizer hosted? How reliable is it?

Visual Website Optimizer is hosted with SoftLayer, one of the most reliable hosting providers out there. Historically, we have seen 99.99% uptime. The static JavaScript is delivered via Amazon Cloudfront and in case of an extremely unlikely event of downtime, our asynchronus JavaScript code is designed to gracefully degrade. Our innovative approach ensures your website doesn't slow down while the Visual Website Optimizer snippet is being loaded.

Visual Website Optimizer has been used on websites of leading companies, banks and government institutes without any hitches.

How secure is Visual Website Optimizer?

We take security of your data very seriously. We support HTTPS and never store your password in plain-text (hash them with a secret salt, for you techies out there) and (honestly!) take care of all essential security measures to ensure optimum protection for your data. Rest assured that VWO is safe and secure. If you have any specific security questions for VWO, feel free to shoot us an email.

Visual Website Optimizer has many enterprise security features as well.

How scalable is Visual Website Optimizer?

We are currently handling billions of requests per month and can easily accommodate millions of additional requests. We have special provisions for high-traffic testing. Make sure you contact us before testing >100,000 visitors per day. For a lesser-traffic site than this, simply sign-up and get started.

How much extra load-time does VWO add to a website?

Our scripts are (g)zipped to minimize their size and heavily cached at the server-side (hello Memcache and APC) and client-side (via browser caching of Javascript). Our servers are pretty fast too (thanks to SoftLayer). So additional load is minimal (around 50ms on average). Even our realtime behavioral targeting, semantic targeting and other advanced analytics technologies are faster than most similar tools. To read more about how our code snippet works, click here.

You can check our ping time statistics here.

Will using Visual Website Optimizer negatively affect your SEO efforts?

Your test pages are created dynamically when someone visits your page. And to display the dynamically generated page, a visitor needs to have both JavaScript and cookies enabled.

The good news is, Googlebot and other search engine crawlers do not support Javascript and cookies. So they only see your original page. As a result, your SEO efforts remain unaffected while using Visual Website Optimizer. For more details, read this.

What is testing?

The aim of website testing is to understand the needs and expectations of your customers. You do this by varying one or more elements of a website on the basis of a hypothesis, which is meant to increase your conversion goal (sales, downloads, clicks, page views, etc). Testing is the method with which you collect data to see if your hypothesis was right.

Testing is a continual process and every website has the scope to improve its conversion rate. Hence, it is very important to come up with good testing ideas, like varying your signup forms, ad copy, calls to actions, trust signals, and others.

What is a conversion?

Conversion is a term used for actions you want visitors to complete on your website. It can be a download, a form submission, payment completion, a video play, visits to a URL, and others. Conversion rate is the percentage of total visitors that come to your website who complete the conversion goal defined by you.

What is a control?

In a scientific experiment, all results are benchmarked with a condition where no change has been made. Only after comparing against original settings, effect of new settings in an experiment can be observed. These original settings in an experiment are called the control.

Similarly, in a test, your existing website design is called the Control and the variation page (new page design) is judged by comparing it to the performance of this Control page.

How does testing works?

You have a hypothesis that you feel will improve conversion rates. You make Variation pages on the basis of this hypothesis to see if it is correct. Your website visitors are randomly served either the control page or one of the variation pages from the test you created.

After the test achieves statistical significance and the test has collected data from a relevant sample size of visitors, the software analyzes which website version resulted in maximum conversions.

Once the winner is declared by your A/B testing software, you can make the winning version permanent on your website for a permanent boost in your conversions.

What is the difference between A/B Testing, Multivariate Testing, and Split URL Testing?

A/B Testing

You change only one element on your website and see which variation results in maximum conversion rate. This element could be button color, button size, headline, call to action or any other element.

Multivariate Testing

You select multiple elements/sections on your website and create variations for all the chosen sections.

For example, on a landing page you may select the call to action button, ad copy and product screenshot as sections. Now for each of the sections, you come up with different variations. All these variations are combined to create unique combinations of your website. If there are 2 sections, one with 5 variations and the other with 3 variations, there would be 5*3 = 15 combinations. All these combinations are pitted against each other to observe the combination which produces the best result. During the test, it is then seen which combination results in maximum conversions.

Split URL Testing

When you wish to test drastically different design changes, you can use page-level testing and split test your traffic between various tests to see which one improves your conversion rate better. For example, you can test your control page against a luxury-focused page and trust-focused page to see which one works better for your target audience.

A B Testing Tool