Posted in A/B Split Testing, Case Studies on March 15th, 2011
We recently did an A/B test on Visual Website Optimizer homepage with an aim to increase signups for our free 30 day trial account. As you can see, our signup form is short (with just a couple of fields), has good social proof (testimonials on the right) and we have a direct link to the signup page in the website header. Thanks to these practices, our signup conversion rate was already quite satisfactory. However, being an A/B testing company, we can’t be content with status quo. So, we decided to optimize our website for more signups.
Homepage as a sweet-spot for optimizing signups
We could have tweaked our signup page in this A/B test. However, as with most websites, the major challenge is to get more visitors to the signup page in the first place. By analyzing our analytics, we realized that our homepage gets many visitors who either simply leave or browse a couple of pages without ever visiting signup form. So, homepage represented tremendous opportunity to increase overall signups. Here is how our original homepage looked like:
Note that above the page fold, we had a Watch Video call to action. Idea was to have let visitors understand the product and then push them to signing up for the trial account. However, note that below the fold we had a button called ‘Start Optimizing Now‘ which directly links to signup page. So far, so good. What do we optimize on this homepage now?
Change the Watch Video button to Start Now button
The first change we wanted to test was the most obvious one. We replaced the Watch Video button to Start Optimizing Now button (which is also present below the fold). Here is how variation looked like:
This increased signups somewhat but the real game-changer was the following change.
Introducing “Popup Forms”: 50% increase in signups
As mentioned earlier in the post, our signup form is fairly short (with only a couple of fields). So why do majority of visitors who finally reach signup page still not end up signing up? The reason is because signing up for any service is a commitment of sorts. Even if signup is free, the act of filling form fields and hitting ‘Register’ button requires great deal of motivation. The usual flow of visitors is usually like homepage -> browse and learn about product -> visit signup page -> do a signup -> try product.
Since signing up requires commitment, we thought of removing explicit “visit to signup page” step by integrating signup as part of browsing the website and trying the product. Implementation wise, here is what we did: on homepage, whenever a visitor clicks on Start Optimizing Now button, we show a “popup signup form” which has only two fields (username and password). Notice that signup form does not require a page reload and appears as part of the homepage itself. See a screenshot of this “popup form” below:
What this did was amazing! It increased signups by 50% as compared our original homepage. We think the increase in signups is primarily because of “popup forms” which have following benefits:
- Less distractions (rest of the page is blackend)
- Only two fields asked for (username and password)
- Page is not reloaded so signup is considered integrated into homepage, not loaded as a separate page. (yes, it matters psychologically!)
What’s next?
Needless to say, we are going to do lot more A/B tests on Visual Website Optimizer homepage and website. Note that this test ran only on homepage, so the next step would be test these “popup forms” site-wide at different places (blog, navigation header, etc.)
Another follow-up task would be to see effect of increased signups on final paid user conversion rate. Right now, the impact on paid conversion rate was not visible as we still need to collect more data. It can certainly be possible that even if we increased trial signups by 50%, actual paid signups didn’t change (or, worse still, actually decreased). This is possible if increased signups are attributed to semi-interested people who may never end up buying a subscription. We used our new revenue tracking feature to see impact of “popup forms” on actual $$ made, however it is still to early to say anything.
If you have any feedback or suggestions on how we can further increase signups, please do let us know by leaving a comment below!
Paras Chopra
CEO and Founder of Wingify by the day, startups, marketing and analytics enthusiast by the afternoon, and a nihilist philosopher/writer by the evening!
Tags
I ♥ Split Testing Blog
Stay up to date
Latest Posts
Latest Tweets
- Interesting results from our new case study - "A/B Testing Between Free and Paid Signups: Sometimes Free is Better" http://t.co/gmd9jyBFwA #
- @JuliaFok Our $129/mo package that gives 30,000 visitors to test seems like a good fit for you. #
- New post on our blog: Extend Your Conversion Funnel Beyond the "Thank-You" Page http://t.co/rEEtlVrxv9 #
- RT @SantiagoEcomm: @Wingify's #SplitTesting blog is great resource to follow for knowledge & case studies on #ConversionRateOptimisation ht… #
- RT @chinchang457: The new toy @wingify >:) http://t.co/NbGLNZ0G61 #
Write for this blog!
We accept high quality articles related to A/B testing and online marketing. Send your proposals to contribute@wingify.com




43 Comments
March 15, 2011
Thanks for sharing your results, this is very insightful.
However, do you have any concerns about using a popup long term – particularly for SEO? From my understanding, Google may tend to ‘disfavour’ sites that collect opt-ins using a fullscreen popup.
What’s your opinion on the SEO impact of a fullscreen popup?
March 15, 2011
@Schwabe: I don’t think there are any SEO implications to this. We show popup after visitor clicks on signup button, not just after page load.
March 16, 2011
Awesome work as always Paras.
Initially, I was a bit surprised at the 50% increase. While I would totally understand a nice lift, 50% is hitting it out of the park.
After thinking about it a bit, I’d have to guess that the “forced focus” (literally removing everything but the form) is stronger than expected.
I’m kinda wondering how much impact the instant form (opposed to loading another page) has.
- Ophir Prusak
March 16, 2011
@Ophir: yep, I think “forced focus” is a big reason for this increase. So it will be interesting to see if eventual paid conversions get increased too!
March 16, 2011
The half that didn’t sign up will now assume you are a spam site though…
March 16, 2011
@Ashley: why do you think so? I am genuinely curious.
March 16, 2011
I think I agree with Ashley… this is a tactic used by Spam websites… and the abhorrent “NO CREDIT CARD REQUIRED” isn’t an encouragement eithre. It is usually a bait to get the visitor’s email ID by hook or crook, which then proceeds to get…ahem…”draped” and pillaged with spam.
March 16, 2011
@John: note that we popup the form only after visitor clicks on Start Now button, not before!
March 16, 2011
If your signup form is only two fields, why not just put that on the homepage instead of requiring people to click the button?
March 16, 2011
@Andy: yep, that’s exactly what we are going to test next!
March 16, 2011
@john I must admit any site that shouts “NO CREDIT CARD REQUIRED” is always a worry to me.
I like the idea of a simple ‘pop up’ form but perhaps simple is the key here. Cutting down on the number of fields may well have been a contributory factor here it may be worth retesting this approach but with more fields rather than email and password.
March 16, 2011
Just for sign up we can use pop up forms.
What about B2B sites where we try to capture leads?
March 16, 2011
@Gunjan: A/B test it on lead-gen sites!
March 16, 2011
What technology, specifically was used to achieve this pop-up?
BleachReport dot com does an incredible job with this feature, but not at the home page. They tailor it to pop-up after spending about 45 seconds on a team page. Then the pop-up is context sensitive.
If you were on the Ohio State Buckeyes page, then the pop-up would be to sign up for newsletter updates about the Buckeyes.
Exceptionally done.
March 16, 2011
Regarding the comments from other users about the “no credit card required” looking spammy, you guys are imposing your personal views on a website that may have a totally different demographic audience than yours. Furthermore, they are sharing a technique that has worked for them, during testing.
This page isn’t about the text in the pop-up, but the functionality used to achieve the results of the pop-up.
March 16, 2011
We used jQuery Facebox plugin for creating the popup.
March 17, 2011
Are you sure the improvements are not related to requiring E-mail + password instead of Full Name (scary) + email + password, and the bad layout of the signup form?
March 17, 2011
@Ricardo: no, not entirely sure. Didn’t isolate a lot of variables in this test.
March 17, 2011
hi,
I love reading about what worked *for you* and how you went about it.
But: what’s with the headline “how to increase signups by 50% using popup forms”? That implies that your recipe:
- relied only on the popup forms
- works for everyone, every site.
Well, it won’t! If one single approach worked, then why would we need a/b testing for a start?
As you point out, you did a lot of changes and got a good result. Excellent – I don’t want to knock that.
By now, thousands of people have read this – partly, no doubt, due to its simple, catchy headline. Exellent – I love it when people pay attention to forms design.
But really! Are we now going to have a spate of articles that say: “pop-ups will increase conversion by 50%”? Answer: probably. And will they be right? Answer: in most cases, definitely not.
At the very least, can I beg you to change your title to: “How we increased signups by 50% by using popups?”
And if you want to be really meticulous, how about “How we increased signups by 50%: using popups, lots of care, and doing a ton of testing?”
best
Caroline Jarrett
March 18, 2011
[...] How to Increase Signups with Popup Forms [...]
March 19, 2011
When I see a pop-up asking me to sign up at a website, I sometimes click the “close” option (if it is very clearly marked) and more often than not, I leave the site and never return. It’s an annoying hassle.
March 25, 2011
I’m going to try to see if I can increase comments on my blog by have a displaying a message when the viewer has reached the comment section.
March 26, 2011
Our landing page “sucks”? Conversion rate is a dismal 3%. What do we need to do to increase the percentage of email submissions?…
Start implementing A/B testing. Take the time to make several layouts, changing button positions, change text, have tests with bold calls to action. Learn from the actual data. Start finding a layout and messaging that starts to push the needle up, the…
March 27, 2011
This is to give everyone an update on my previous comment on to see if this “Pop up” form would work for my website.
Result: Successful.
More info:
My goal for this short experiment was to decrease the click/vote ratio for my articles posted on dzone.com.
After 10 days, my highest rated article received a click/vote of 270/7 = 38.6 clicks to get one vote.
But in just two days, using a pop-up message that appeared at the top of screen on, another article gave me click/vote ratio of 147/12 = 12.25 clicks to get one vote.
However, a few problems with my experiment is that I used different articles to compare results and also my small sample size.
If you disregard those two drawbacks, then I would I say this “pop-up” technique works and does provide an average of 50% increase in user involvement.
For those wondering, a popup message of “Please leave a comment before you go.(click to exit)” was displayed at the top of the screen when the user hovered over the content. The popup message was made with jquery-topbar.js.
Thanks for the article. I’m sure this site has more useful information.
March 27, 2011
@Larry: thanks for sharing that. I am glad the same technique worked for your site. Though do a proper A/B test to verify it :)
April 22, 2011
Hi Paras,
Thanks for sharing this with us.
I’ll remember to use it whenever I ask for an opt-in on the home page.
Best,
E
April 23, 2011
How would I achieve this effect using visualwebsiteoptimizer? Could I do a regular A/B test and put the jquery code in one of the variations? Or would it be better create two separate versions of the page on my own and use the Split URL test feature?
April 23, 2011
@Brandon: we used default A/B testing, so you can use that too. Contact us on support if you are not able to figure out how to do it, will be glad to help.
April 26, 2011
[...] combined with a short form, using a modal window can increase signups by 50% or more (as it did for Visual Website Optimizer). The modal window also has the benefit of decreasing distractions on the signup page, as it shades [...]
May 20, 2011
Some of you guys aren’t getting it. It DOESN’T MATTER what you think – really. What matters is what most people who visit the site think, and, not what you THINK they think! :-)
That’s why you should A/B test instead of “thinking.”
A/B Test, scrub, rinse, repeat.
June 17, 2011
Have you also tested offering some kind of incentive along with your offer? Example: a free ebook (as long as it’s relevant).
I’ve been noticing more companies using this technique. For example, Citrix is using a book by Jill Konrath (a real, published book, in pdf format) as the incentive.
Thoughts?
June 17, 2011
@John: giving stuff away for free in order to incentivize people signing up for trial is an interesting strategy. But I think it incentivizes people in a wrong way — we want people to be motivated to try out the product because of its potential usefulness. With a freebie such as a book, people may signup just because of book and may ignore the application completely. But, again, it’s a hypothesis — real answer can be found after A/B testing it.
October 3, 2011
Hi being working in b2b industory, the tast that i have been assigned is to increase sign ups, recently we have practices of pope-up to sign ups and it works great.. can you please suggest me any other ideas which could increase our sign ups…
Thanks
December 23, 2011
[...] Notice what happens when the user clicks on the big blue button: The email popup window on Visual Website Optimizer’s home page is initiated by the user. This offers a very different user experience than does the popup that startles the user when it appears “out of nowhere”. Interestingly, Visual Website Optimizer experienced a 50% increase in sign ups when they implemented this user-initiated email popup form versus the user going to a separate page to sign up—read the full article here. [...]
February 3, 2012
[...] Use popup sign-up forms. Popups are known to boost email subscriptions, but Visual Website Optimizer got 50% more signups thanks to a popup signup form. [...]
March 2, 2012
[...] How to increase signups by 50% using “popup forms” This increased signups somewhat but the real game-changer was the following change. [...]
April 2, 2012
[...] but proven to increase subscribers (Visual Website Optimizer found a 50% increase in sign-ups). I personally loathe the pop-up so make sure to consider how much you may be turning off some of [...]
April 3, 2012
What about A/B testing the actual popup. For example, testing the popup headline or seeing if you can add another field to the popup.
Have you tried this? Would it be possible if the popup was JavaScript and didn’t have it’s own unique URL to run a split test with?
Would love to hear your thoughts on how this could be accomplished :)
April 4, 2012
@Jon: yes, we should have tried optimizing the popup itself. We can test JavaScript popups using VWO advanced capabilities (JavaScript and jQuery based API), but we haven’t gotten around testing it yet.
April 17, 2012
[...] myśli o formularzu sign-up, przychodzi nam do głowy nowa podstrona naszego serwisu. Jak pokazują testy Visual Website Optimizer, często lepiej jest pokazać użytkownikowi toplayer, niż przekiero…. Visual Website Optimizer zastosował toplayer zamiast przekierowania uzytkownika na kolejną [...]
April 23, 2012
50% increase in sign ups is really awesome, i’d agree it’s probably because the rest of the page is blacked out so it’s really clear what to do. A lot of websites are so packed full of buttons, graphics, offers and info it can be difficult to know where to go and what to click on! The key is to keep things simple!
April 29, 2012
Hi guys! May I propose you another one variant for testing your 30 Days Free Trial proposition derived from my usecase? ;)
I am interested in your service, but not right now. So I delayed signing up for free trial until we launch my startup (planned for end of July 2012). But in July I may not remember your URL and, choosing the service provider for multivariant and A/B testing, can choose another one.
So you can try to offer to sifn up for free trial, but ask users when they want to start their 30 days free trial (and tell them they can delay this date later). So you can aquire more interested customers and communicate with them via e-mail with your use cases until they start to use your services.
April 29, 2012
@Arthur: that’s a great idea. We will definitely test it.
Sorry, the comment form is closed at this time. To start a discussion, please email info@wingify.com.
RSS feed for comments on this post.