Two weeks back we launched a brand new interface for Visual Website Optimizer which, thankfully, was very well received by our customers. They love how simple we have made creating A/B and multivariate tests. They also love the redesigned, detailed (conversion rate, revenue and heatmap) reports. Today, we are proud to release an update to the new interface: a comprehensive test management interface!
Why do we need test management? Well, some of our customers do very large scale testing. They have hundreds of tests in their account with many that are currently live and running. The plethora of (A/B, split, multivariate or heatmap tests) made it somewhat difficult to make sense of the list of tests. So based on user feedback, we have come up with a new test management interface. Have a look at the screenshot:
There are number of new test management features but following are the major ones:
We hope you like the new features. As usual, if you have any feedback for us, we’re all ears!
We have a lot of A/B testing case studies on our website now. Most split testing case studies describe a single change and discuss the increase of conversion rate it caused. However, this post is a compilation 3 different A/B tests on a single website. This one is a great case study to see how some companies take multi-pronged approach to successfully optimize different steps of the funnel. These A/B tests were conducted by one of our customers ABNRegistrations.com who provide quality ABN registration (Australian Business Number).
The first split test that ABNRegistrations.com did was very simple. On their order page they describe three steps of the ordering process. For the variation, they simply added green ticks next to these steps. Here’s how original order page looked like:
Original order page
Here’s how variation looked like:
Variation (with green ticks)
Result: 2% increase in conversions
As you can see, the change was very trivial. Results were also trivial: a 2% increase in conversions (note that even though increase was small, it was statistically significant). However, this small increase did not deter ABNRegistrations.com. In their own words, they resolved to achieve perfect conversion rate. Here’s what they said:
At a recent staff meeting we set a goal of a 100% conversion rate. We know we won’t achieve this but it provides a focus on ‘continual improvement’. In other words, we don’t set a goal for each test, we just look to improve each time
In the next split test, they tried changing their headline. Originally, they promised a response time of 24 hours. They tested a version which promised a response time of 4 hours. (Of course they weren’t bluffing as they had to change their internal systems to honor this guarantee). Here’s how original headline looked like:
Original headline (with 24 hour response guarantee)
Here’s how variation looked like:
Variation headline (with 4 hour response guarantee)
Result: 10% increase in conversion rate
This (apparently) simple change boosted their conversions by 10%. What’s amazing is that as a side effect of this 4 hour response guarantee, ABNregistrations.com increased their overall quality of service.
On their order page, they tested a better looking order form. Here’s a screenshot:
Improved and better looking order form
Result: 25% increase in conversions
This phenomenal increase occured thanks to complete redesign of signup form. They increased conversion rate from 30.88% to 38.88% (an increase of 25%). You can see this another case study which shows how redesign of sales page increased sales by 20%.
This case study teaches us that all aspects of conversion funnel can be optimized. From headline to text copy to order form, there is nothing on a website that cannot be tested. So it is important to always keep testing. For ABNRegisterations.com, split testing is ingrained in their culture. The biggest lesson they learned was (in their own words):
Don’t assume anything without testing it. Visual aids/images are just as important as text. Simple changes can make a difference.
When asked how useful was Visual Website Optimizer for their split testing, here is what they had to say:
VWO was incredibly valuable in helping us achieve this, both in terms of the ease of use of the system AND the idea for using the green ticks. I recommend that other business owners and managers take the time to read through VWO’s ideas and brainstorm the things that they might be able to try in a split test. Also, testing should never stop. We are already setting up our next test for the same page.
Needless to say, we are very happy for their success and hope they will continue split testing for many more months to come!
Today is a very proud day for us primarily due to two reasons: a) we are launching completely revamped interface of Visual Website Optimizer that is packed with loads of new features; b) we have achieved 10,000 total signups (trial + paid).
This new interface has been in development for about 3 months and it completely changes the look-and-feel of the application. Focus of the new interface has been to make creating and running A/B (or multivariate or split) tests even easier (and snappier). The new (tabbed) reports allows you to intuitively interpret test results. Know which variations are performing better and which ones are performing worse on goals such as revenue, conversion rate, visitor engagement, etc. Okay, enough of talking. Watch a short video of the new interface:
It is not just the interface that has changed. We have added number of features alongside. Sample a few new features:
There are many other subtle improvements that we will leave to you to discover.
Following screenshots will show you different aspects of Visual Website Optimizer’s new interface.
Sample Report
Thumbnail Previews
Heatmap
Testing Options
Plugins / Integrations
Send report via email
We begin our journey about a year ago (in May 2010) when we launched Visual Website Optimizer paid plans. Today we are psyched to announce that we have achieved a total of 10,000 signups (free trial + paid). Our customers include individual bloggers, small businesses and all the way to large enterprises such as Mircosoft, AMD, Groupon and FourSquare. Our revenue has been growing by 10-15% consistently for last one year and thanks to increased userbase our servers are now handling 2.2+ billion pageviews per month. We are proudly self-funded and are extremely profitable. Next phase is to expand the team and hence we are actively hiring engineers and sales guys. If you want to shake the website optimization world, join us!
This new interface was driven entirely from user feedback and we thank all early adopters for helping us fix kinks before release (including but of course not limited to Janco Klijnstra from Traffic4U, Ahmad Rahman from AMD, Lucy Spencer from LoveFilm, Nate Ende from Trinity Insight and Karl Blanks from Conversion Rate Experts) . We are incredibly lucky to have them as customers as they take their precious time to help us improve Visual Website Optimizer.
Of course, we are far from done in our mission to make A/B testing fun, easy and profitable. We would love to hear your feedback on new interface and how we can further improve Visual Website Optimizer.
If you want to try out the new interface, simply signup for our no commitments free 30 day trial and let us know what you think!
This is a guest post from Jeremy Reeves, who is a freelance copywriter obsessed with split testing everything from emails, to landing pages, salesletters and upsell processes. You can find more information about Jeremy’s split-testing services.
Let’s face it.
Testing kind of sucks. For most people anyway.
It’s hard to learn… takes a lot of time… and typically requires you to hire a split-testing service provider like me to get the job done right (unless you’re a testing-obsessed fanatic like I am and love doing it!).
But – you also know the financial rewards of it or you wouldn’t be reading this blog 
And you know that testing can be the single most profitable investment you can make in your business.
But in order to make that happen… you or the person doing split-testing for your business must be aware of the BIGGEST mistakes I see people making when split-testing.
So in this post I’m going to reveal top 7 split-testing blunders your MUST avoid if you want maximum results in your business.
This is a long and information-packed post… so grab a cup of coffee… sit back… and let’s get started.
What are “whispers” and what are “screams”?
Whispers – These are tests such as changing the font, changing the background color, changing the color of a button, testing something in the footer or very low on the page… and basically anything that has no (or very little) chance of getting you a BIG boost in conversions.
(And yes, I realize that “sometimes” some of these things can give decent boosts – but it’s very rare)
Screams – These are tests that are likely to either completely BOMB… or hit a grand-slam home run. They’re the ones that make you anxious to test, but can produce mid double-digit…sometimes triple-digit improvements. Screams are things like getting your copy completely rewritten, testing out a radically different design, testing radically different price points (e.g. $47 vs. $97 instead of $49 vs. $49.95), or coming up with an unheard of and radical guarantee.
Do screams take longer? Yes. Do they require more energy and resources from you? Yes. But is it worth putting money into hiring someone to test radically new copy or a design? Absolutely!
This is one that 99% of marketers (especially online) are guilty of. They “assume” something will work, so they don’t test it.
Here’s a hint.
Don’t assume anything!
I personally conduct split-tests every single day for my clients, and I’m STILL amazed at just how “weird” some of the tests come out to be.
Think your headline sounds “perfect” and nothing can beat it? I guarantee you – some other headline out there floating around in the ethers of your mind (or your copywriters mind) CAN beat it.
Think your pricing matches your market perfectly? I bet not. You don’t know until you test.
In love with your product name? I bet if you split-tested it using Adwords (which is a neat little trick, and easy to do) – you could find one that converts better. After all, when it comes to testing it’s about money, not ego. Go with whatever gets you the highest conversions.
I’m assuming you already have Visual Website Optimizer (if you don’t, you’re nuts)… so I won’t pound this one in too much as you’re probably more familiar with testing than most marketers.
Here’s the mistake most people make.
They think “testing” goes like this.
Step 1 – You watch the conversions of “X” page for 1 week… or for 100 conversions (or whatever other metric).
Step 2 – You then switch that page and replace it with the new test you’re doing… and watch those conversions for 1 week, 100 conversions, etc.
Step 3 – You compare the data, see which had a higher conversion, and you now have a winner!
{Insert obnoxiously loud “eerrrrr” sound}
That’s NOT how you should be testing.
For a test to be accurate, you MUST have all variations running at the exact same time frame, to the same audience.
It’s that simple.
Here’s another one I see TONS of people make. Mostly, it’s an “ego thing” because they want to brag about the huge increase they got.
Here’s what I mean.
I’ve ran plenty of tests where one variation was the control, hitting 95% confidence, after just a handful of conversions (let’s just say 10, for example). Let’s also assume the losing variation only had 4 conversions.
Does that mean the winner actually beat the control by 250%?
Absolutely not!
I’ve seen tests like this where the losing version came back to beat the one that was winning in the beginning.
Why?
Because you need to run your tests long enough to get accurate results! You need to get past what I call “random coin syndrome”.
What I mean is this.
A regular coin should flip 50/50 over time. But I want you to test this yourself. Go get a coin and flip it 20 times. In most cases, you’ll get very uneven results. Something like 15 heads and only 5 tails. Or 12 tails and 8 heads.
And the SAME thing happens with testing. In the beginning, with under 50 conversions or so (this is a rough number and depends on the conversion number), a lot of the results happen by chance.
My advice is this.
If you’re looking to be accurate by making sure your tests are valid – try to get at least 50 conversions or so before making any decisions – EVEN if you have a winner after 20 conversions.
Oh yes, there’s a flip side to what I was just talking about!
Running your test for too LONG is a killer of momentum (and sales)
Here’s an example.
Let’s say you’re testing a “whisper” as I mentioned earlier. In this example we’ll say you’re testing your background color.
5 weeks after you launch the test… the variations have only a 2% difference in conversion.
What do you do?
Here’s my advice.
Depending on how much income that product is bringing in, 2% may or may not make a big difference to your bottom line. If it’s NOT, I highly suggest going with whatever is winning at the moment (even if it’s not at 95%+ confidence) and setting up a new test that is likely to give you a bigger bump.
Bottom line: Don’t get addicted to testing results. If a test goes on for longer than normal without a nice increase in conversions, scrap that test and move on.
(The same is also true for lousy products, by the way. Get your emotions out of the equation!)
Learning your market from your tests is CRITICAL to long-term testing success. It will help you dive into the mind of your prospect and give you a “knack” for knowing what they want.
Let me explain.
When I write copy or do split-testing for my clients… the first step I do is RESEARCH. In this phase I basically submerge myself into the middle of the market and try to understand their wants, needs and desires.
I then use what I discovered in my research to write about in the copy, or to create split-tests with.
And while that’s great and works like gangbusters… it works even BETTER when you can do your research based off what people are actually doing with their wallets.
Here are just a few examples of what you can learn about your customers.
Video: If you have several products, consider testing video against text. Test several variations of video/text together (AKA a “hybrid”). If you notice that video wins… you can then roll that test out and test it on all your other products. In “most” cases, you’ll have a winner each time.
Guarantees: Test several guarantees for one of your products. Let’s say you have 5 products and all of them have 30 guarantees. Change the guarantee to 90 days for one of them, and see if you get an increase in conversions (you will). Let’s just say you get a 10% bump. Well… if you then roll that out into the other 5 products, you can count on a 10% bump on each of those as well… giving you a total of 50% in total increased conversions spread out throughout your site.
Copy Styles : You have several options when it comes to copy. Hypey versus soft. Long form versus short form. Professional versus personal. And while you may “think” you know what type of copy works the best, it’s impossible to know until you test it. But here’s a tip to get started. In MOST cases, a mix of hype versus soft… long form… and personal… works the best. However this all depends on your business, product, market and offer… so test it. Once you figure out the perfect formula – use that for the rest of your site!
Note: If you don’t think long form copy works anymore, read this post I wrote about the anatomy of a salesletter after you’re done with this article.
Want to know if your ego is getting in the way of your testing results?
Take this litmus test.
Has anyone ever suggested running a test and you said “no, that won’t work”… or “no, my customers won’t like that”… or “no, that will make us look bad”…?
If so, your ego is getting in the way.
Sometimes, that’s fine.
If you aren’t trying to maximize profits and would rather secure a specific mental image in the mind of your prospect (such as a certain design style, etc.), that’s your decision.
A perfect example of this is using “doodles” in your copy. These range anywhere from handwritten notes in the margins… to using a crayon on a piece of direct mail.
MOST people would think they wouldn’t work. That “their customer is too ’sophisticated’ for that”. Yet a company I know tested several variations of a letter they were sending out… and then tested one that was in the voice of the owners young son… written in crayon.
Keep in mind, this letter was going to a clientele that were “sophisticated”.
So how did it do?
Well, let’s just say… it was the single most profitable letter they EVER sent out in the lifetime of their business
Sometimes it’s hard letting go over your ego, I get that. But it’s necessary.
Hopefully this article has helped you ramp up your testing efforts. Or, if you’re hiring someone and having them do it… make sure they understand all of these blunders as well. It can literally make the difference between grand-slams and bankruptcy.
