Recently on Hacker News, someone commented that A/B testing has become like snake oil, making grand promises to be a panacea for increasing conversion rates. This comment troubled me not because the commenter was wrong (he was!) but because how certain people view A/B testing. Let me put this straight and clear:
In fact, we recently featured a guest post by Noah of Appsumo who revealed that only 1 out of 8 split tests that they run produce any actionable results. Yep, read that once again: only 1 out of 8 split tests work. Snake oil guarantees to be a cure for all diseases; A/B testing has no such guarantees.
I think the reason why some people would view split testing suspiciously is because we release a lot of A/B testing case studies. In fact, with a user base so large (6000+ at the time of writing), we see many of our customers getting good results all the time and they if want to share those results with the world, we publish them on our blog.
So, when one week we publish a case study titled 3 dead-simple changes increased sales by 15% and the next week we publish another case study titled How WriteWork.com increased sales by 50%, it naturally makes some people assume using Visual Website Optimizer to do A/B testing would ensure a growth in sales. No, using Visual Website Optimizer won’t guarantee anything. It is just an A/B testing tool (albeit, a very good one!) but the real work done is always by the craftsman, not the tool s/he uses.
Most success stories or case studies (be it about A/B testing, getting rich or losing weight) that you read about on the Internet suffer from something known in psychology as survivorship bias. In context of A/B testing, what it means is simple:
If you run 100 different A/B tests and only 1 of them produces good results, you are only going to publish about that one good result and not the 99 other unsuccessful results.
Obviously, 99 case studies about how A/B testing did not produce any results in unexciting and will bore our blog readers. Hence, this post is a short reminder about all those case studies that don’t produce results.
The reason we publish A/B testing case studies is to show potential of what can be achieved using A/B testing. It is similar to holding an art exhibition for Picasso paintings which inspires people to appreciate the work and perhaps pick up brush themselves to start painting hoping to achieve level of success that Picasso achieved. To reiterate, just as picking up a brush and canvas does not guarantee good art, doing A/B testing does not guarantee increase in sales.
In fact, specifically to A/B tests, replicating them almost never works. So, if you read a case study of how video increases conversions by 46% and then implement it on your site hoping for the same magical increase, it may or may not happen. What worked for them may not work for you! So, why publish case studies at all? Here’s why:
A/B testing case studies give ideas on what you can test on your website
Yep, that’s all there is to A/B testing case studies.
And, yes, I hope I convinced you that A/B testing is not snake oil!
We recently did an A/B test on Visual Website Optimizer homepage with an aim to increase signups for our free 30 day trial account. As you can see, our signup form is short (with just a couple of fields), has good social proof (testimonials on the right) and we have a direct link to the signup page in the website header. Thanks to these practices, our signup conversion rate was already quite satisfactory. However, being an A/B testing company, we can’t be content with status quo. So, we decided to optimize our website for more signups.
We could have tweaked our signup page in this A/B test. However, as with most websites, the major challenge is to get more visitors to the signup page in the first place. By analyzing our analytics, we realized that our homepage gets many visitors who either simply leave or browse a couple of pages without ever visiting signup form. So, homepage represented tremendous opportunity to increase overall signups. Here is how our original homepage looked like:
Note that above the page fold, we had a Watch Video call to action. Idea was to have let visitors understand the product and then push them to signing up for the trial account. However, note that below the fold we had a button called ‘Start Optimizing Now‘ which directly links to signup page. So far, so good. What do we optimize on this homepage now?
The first change we wanted to test was the most obvious one. We replaced the Watch Video button to Start Optimizing Now button (which is also present below the fold). Here is how variation looked like:
This increased signups somewhat but the real game-changer was the following change.
As mentioned earlier in the post, our signup form is fairly short (with only a couple of fields). So why do majority of visitors who finally reach signup page still not end up signing up? The reason is because signing up for any service is a commitment of sorts. Even if signup is free, the act of filling form fields and hitting ‘Register’ button requires great deal of motivation. The usual flow of visitors is usually like homepage -> browse and learn about product -> visit signup page -> do a signup -> try product.
Since signing up requires commitment, we thought of removing explicit “visit to signup page” step by integrating signup as part of browsing the website and trying the product. Implementation wise, here is what we did: on homepage, whenever a visitor clicks on Start Optimizing Now button, we show a “popup signup form” which has only two fields (username and password). Notice that signup form does not require a page reload and appears as part of the homepage itself. See a screenshot of this “popup form” below:
What this did was amazing! It increased signups by 50% as compared our original homepage. We think the increase in signups is primarily because of “popup forms” which have following benefits:
Needless to say, we are going to do lot more A/B tests on Visual Website Optimizer homepage and website. Note that this test ran only on homepage, so the next step would be test these “popup forms” site-wide at different places (blog, navigation header, etc.)
Another follow-up task would be to see effect of increased signups on final paid user conversion rate. Right now, the impact on paid conversion rate was not visible as we still need to collect more data. It can certainly be possible that even if we increased trial signups by 50%, actual paid signups didn’t change (or, worse still, actually decreased). This is possible if increased signups are attributed to semi-interested people who may never end up buying a subscription. We used our new revenue tracking feature to see impact of “popup forms” on actual $$ made, however it is still to early to say anything.
If you have any feedback or suggestions on how we can further increase signups, please do let us know by leaving a comment below!
In the past, you must have come across one of those long sales page that never seem to end. Most designers hate long sales letters because they contain heaps of text but apparently lack on aesthetics front. Some people go extreme and even call such sales pages as evil forms of internet marketing. Truth be told: long sales letter pages are based on learnings from direct marketing and catalog industry, so there is nothing inherently evil about them. In fact, many marketers still use long form of sales pages because they convert visitors like crazy!
This is the 3rd article in the series of interviews and guest posts we are running on this blog regarding A/B testing and conversion rate optimization. In 1st article, we interviewed Oli from Unbounce on Landing Pages Best Practices. In 2nd article, Noah Kagan of Appsumo shared his A/B testing tips.
In this post, we interview Freelance copywriter Jeremy Reeves, who is a direct response copywriter and marketing consultant. He specializes in long sales letter copywriting. His unique and powerful marketing strategies, combined with results-getting copy have earned his clients millions of dollars. To learn more about Jeremy and see how he can grow your business, head over to www.JeremyReeves.com/freelance-copywriter.html Following is the interview with him.
Question: What are the most common elements of a long sales-letter type webpage? Can you break them down from top-to-bottom?
Step 1 is the main headline area. This includes the pre-head, main headline and subhead. This is the point where you want to instantly grab their attention and basically – tell them WHY they’re about to read the page. Tell them what’s in it for them, and do so in an exciting way.
- Click the image above for full screenshot -
This is also known as the “Deck” copy”.
Next is the opening words, also knows as the “lead” or “introduction”. This is where I like to get in touch with them emotionally. Depending on the market, product and hook I found when doing my research, this could be any of the major emotions such as anger, guilt, fear, pride, etc…
Next comes the body copy…
The body copy would include areas such as setting them up for the sale, revealing the solution (e.g. why it’s special, how it works, how it’s better than others, etc.)… why they should be listening to you… the benefits of the product… social proof, etc.
(This is the largest section of the copy)
Next, you make your offer. You tell them exactly what they’ll be getting, what the price is, the unique value proposition of whatever it is that you’re selling (e.g. why it’s worth more than what they’re spending on it)… the guarantee… and all that good stuff.
Basically – you’re giving them “X” in exchange for “X” dollars.
At the end of the offer section of the copy, I like to put in an order box containing a summary of everything you just said. That way, skimmers can see the “whole package” at a glance. You show them every single item they’re getting, how much everything is worth… and why they’re paying X% less than what it’s worth.
After this, comes the “close” where you can do many things including injecting scarcity into the offer (but PLEASE don’t inject false scarcity)… having them think about the consequences of NOT taking action, reminding them of the benefits/value/guarantee/scarcity, etc.
The close would basically be everything after the order form on the page – including the P.S.
And that’s basically it! I can break it down into even more chunks, but it depends on MANY factors so it would only confuse people who don’t already understand it.
Question: What makes such pages so effective? Which psychology tricks do these pages adopt to hook the reader?
What makes them so effective is that, if setup the RIGHT way… you can hook into the personality styles of all types of readers.
There are basically 3 types of readers.
If you check out the image I talked about above… you’ll see how I break up the pages into “subheads”, which I’m sure you’re familiar with. What you want to do for long-form sales-letters is make is so that someone can literally read the headline, and the subheads… and have the minimum information they need to make a buying decision.
But that’s a whole other issue in it’s own
The best thing a person can do is turn skimmers, into jumpers… and jumpers… into book worms. That’s why it’s important to have fun, exciting, engaging copy. Just because it’s long, doesn’t mean it’s necessary – or even good.
That’s one thing most long-form copywriters don’t understand.
As for psychology tricks, there are a million of them. To be honest, I personally don’t worry about psychology tricks all that much. I use them, but sparingly because if you use them too much – people start to notice.
Psychology tricks are great for selling books, but simply getting to know and get a deep understanding of your customer is what sells in real life. And, of course… adding the “essentials” like proof, credibility, a great offer, bonding with your prospect, engaging copy, etc.
Question: I have heard that these pages draw inspiration from direct mail campaigns and infomercials. Is that true? What lessons can we learn from those two mediums and apply on the web?
Yep, absolutely!
One of the biggest things I’ve ever learned from infomercials is that it’s ALL about the product. If you’ve ever seen Billy Mays commercials – those products sell you in a split-second. They use something called “demonstrability” which is basically – showing the product in use.
For example in one of Billys infomercials they were selling a small saw, and to demonstrate it – they literally cut a car in half. If you’re not sold after that – I don’t know what will sell you!
Demonstrability is amazingly powerful, and something very few Internet Marketers do these days.
As for direct mail, the biggest lesson is that no matter what you think, or what anybody tells you… PEOPLE READ WHAT THEY’RE INTERESTED IN.
I’ll get into this more in the next question.
Another awesome example is Schlitz beer. A few decades ago, they were having a hard time getting noticed. I think they were ranked #6 (this was when there were only a few dozen breweries). One of the most famous copywriters who ever lived… wrote a longer-form ad about the process of making their own beer… and they not only hit #1… they got so busy they couldn’t keep up with the demand!
Question: Is longer sales-page page always the better? Or is there an ideal length?
Ok, here’s a biggie.
It’s kind of funny that many long-form copywriters think longer copy will win in every single case… and on the flip side… a lot of “Ecommerce” guys think that short copy will always win.
… and they’re BOTH wrong.
Here’s the “secret” formula if you will.
Tell them everything that you think will push them one step closer to buying your product… and not a single word more. And do so in a fun, engaging way that will connect with YOUR audience.
Also, in general – the lower of a price the product is, the less you have to say (because it’s a more “impulsive” buy).
Trust me – if people are interested in your product, they’ll read for a LONG time before making a decision if they need to. I know guys who have written 60 (yes, 60) page letters to sell expensive coaching and seminar packages.
Some of the biggest financial and health companies in the world use 32 page magalogs to sell their prospects.
… so don’t tell me people don’t read anymore, because they do – and always will.
Here’s the key.
If you find that people aren’t reading your copy and feel it’s because it’s “too long”… try hiring a great copywriter and look at the difference. The difference isn’t that it’s “too long”… it’s that it’s not engaging enough.
I have a salesletter right now for a client selling a $300 product where his “per visitor” value is $10.25. (So basically… for each person that hits his page, he makes $10.25). And that page is probably 12-15 pages long or so… maybe even longer.
… and no, it’s not an information product. It’s a physical product that gets shipped to your house and weighs like 10 pounds.
Question: Are there any examples of best sales-letter pages on the web and what you like about them?
Well instead of just pointing out some letters I like, let me show you another one of mine simply because I also have statistical data to back up that it beat the shorter version (by just a hair under 50% with a 99% confidence interval).
http://www.losethebackpain.com/inversion3.html
And as you can tell this page is pretty long. I’m not sure exactly how long but it’s definitely a “long-form” salesletter.
We tested this version against a control this client had which was much shorter (it just had a video, a few bullets, summary of the offer, etc.)
Question: How important is copywriting on a sales page?
The funny thing is… even though I’m a copywriter – I’m willing to admit that copywriting is NOT the most important thing on a sales-page.
What’s the most important?
The OFFER.
The offer consists of the product you’re giving them, any bonuses, the price, returns/shipping policy, guarantee… and anything else included. In simple terms…
… they’re giving you “X” dollars and in return they are receiving “X”.
If you look at this example selling an “inversion table” you’ll see that I layout the entire offer right inside the order box. This is for the people who want to get straight to the offer to see what it is. Instead of simply saying “you get an inversion table for $299”… I list out ALL the features, the benefits, an image, as well as everything else in the offer.
The best copy in the world can’t sell a crappy product, but mediocre copy can sell a fantastic product. The REAL magic happens when you combine top-notch copy… with a top-notch offer. That’s why whenever I get a new client I always talk about the offer with them and help them improve it – before I write a single word of copy.
… and my version that you see is beating his original version by just a hair under 50%. I don’t know about you… but a 50% increase on a $300 product typically means a pretty big jump in profit
The best part is – I haven’t even gotten a chance to test this page yet. By the time I’m done testing out 4-5 different “themes” (the overall “big idea” of the page)… I’ll likely bump it up another 50%.
Question: What are your favorite copywriting tips and techniques?
The best technique you can ever use in copy is simply understanding the dominant emotions going on inside your prospects mind at the exact moment he/she is reading your copy. If you can understand that… it paves the way for every single word on the page and the copy flows like water.
It helps you understand how to phrase things, what kind of offer to give them, what to price your products at, what benefits to hit on, how aggressive or subtle the copy should be… and literally everything else you need to know.
Question: I have talked to couple of UX and web designers and they absolutely hate these long sales-letter pages. They find it scammy and cheating naive web surfers. What are your thoughts about this?
My thought is… they’ve never tested it.
I don’t understand how people can find it “scammy” or “cheating”. You’re giving the customer ALL the information they need to know, as well as giving them the choice to either skim the page to get the gist, and then make a buying decision… or read it word for word.
Also… if they think this – it’s also a sign that the copywriter sucked. Great copy is engaging and entertaining as I mentioned earlier… but it should also be EDUCATIONAL. The goal is to get people to read it word for word.
If you don’t think people read it word for word… again you’ve never tested it (or your copy sucks). Just this morning, a prospect for a piece of copy I wrote noticed a TINY spelling mistake buried on about my 10th page of copy.
… which made me glad because it shows that people are truly interested in the content and reading it thoroughly.
Question: Do you think “sophisticated” web designers lose a lot because they are not ready to embrace techniques employed by long sales letter pages?
Absolutely. In fact, I’d be willing to bet that if they started testing long-form salesletters… they’d very soon be able to buy themselves a new house.
(Depending on how big the business is already, of course. Maybe they could only get a new car
The HUGE majority of sites on the Internet today are what I call “OED” (owner-ego-driven). Let’s just put it this way to make it simple. If you look at your site and the first thing you think is “wow – that’s great looking!”
… you’re probably losing a TON of money (in most cases, not all).
Take my example earlier. I took my client from a short letter to a long letter… and gave him just about a 50% increase in profits for that product.
I’m still working with that client to re-do every salesletter on his site… but what if each new letter I write increases the profits to that product by 50%? Well… he then not only has 50% more profit for that product, but can then dump those profits into more advertising, more affiliates, R&D, new products, etc. etc.
Another client I just finished a project for ended up getting a $3 EPC for a $47 digital/$77 physical product.
(Comparable products that I didn’t write were about $1 EPC)
I’m not touting my horn here – my point is – they were ALL long-form salesletters.
So that about wraps it up. To whoever is reading this – I really hope it helped you. If I offended you… I apologize, but I hate to see people losing thousands of dollars per month (or even day) simply because they’re not A/B testing things they should be testing. (Editor’s note: Jeremy uses our product Visual Website Optimizer or A/B split testing his copy)
Hopefully this will change your mind about long-form copy and show you that in the VAST majority of cases (in particular information products and high priced products/services)… it outperforms short copy.
This does depend on the website, and the market, and the individual page… but either way – it’s absolutely 100% something to test.
Freelance copywriter Jeremy Reeves is a marketing consultant whose unique and powerful marketing strategies, combined with results-getting copy have earned his clients millions of dollars. To learn more about Jeremy and see how he can grow your business, head over to www.JeremyReeves.com/freelance-copywriter.html
Editor’s note: Hope you liked the interview. For next articles in this series, if you know someone whom we can interview or want to contribute a guest post yourself, please get in touch (paras@wingify.com).
