Real-world A/B testing proves that shorter copy converts better

I discovered Google Content Experiments a couple of weeks ago. If, like me, you’re already using Google Analytics and tracking conversion rates, it’s incredibly easy to set up A/B tests.

Hypothesis

I wanted to test a common hypothesis about short copy. For example, Jakob Nielsen’s research says that visitors read about 20% of the text on a page and that a 50% reduction in word count produces a 58% improvement in a page’s usability.

Prediction

I was testing a page on the Turbine tour. It’s a landing page for Google Ads so it needs to convert well to reduce the cost of customer acquisition. The call to action is a free trial signup on the page itself. I expected a small uplift in the number of conversions on the page with the shorter text.

Experiment

So, I tried two variants of the page, which is about online purchase ordering. The original had 609 words. The revised version had just 148 words. You can see the dramatic difference in length and word density in these two screen shots.

TurbineHQ.com short text screen thumbnail TurbineHQ.com long text screen thumbnail

The experiment ran for about two weeks before it had enough traffic to declare a winner. Here’s a graph of conversions for the two different variants over the time.

image

Analysis

The results are conclusive. Short text beat long text 11 conversions to 5 and a conversion rate of 5.95% to 2.53%. This is an improvement of 135% overall.

So, do you want to double your website’s conversion rate? Here’s how: halve the copy and then halve it again.

Do your own experiments

In WordPress, this is my recipe for A/B testing:

  1. Install the Google Content Experiments plugin.
  2. Create one or more alternative pages to the test control page (the page you’re experimenting on)
  3. Set up the experiment in Google Analytics (look in the Contents menu for Experiments and click Create experiment)
  4. Follow the instructions
  5. Take the short bit of code Analytics gives you and drop it into the box on the control page provided by Google Control Experiments.
  6. Bob’s your aunt’s husband.
  7. Once the experiment is finished, you’ll need to change the URL of the winning page so that it replaces the control page URL (or of course leave it alone if the control won!). But remember to redirect the losing page URLs in case anyone bookmarked them. I use the Redirection plugin for this.

, , , , , , ,

5 Responses to Real-world A/B testing proves that shorter copy converts better

  1. Peter Cooper September 13, 2012 at 3:07 am #

    I enjoyed this post but I wanna go “whoooooaaa, hold on a minute” to “Here’s how: halve the copy and then halve it again.”

    I’m a huge fan of split testing and do it a lot and sometimes long content will beat out short, sometimes short will beat out long.. that’s the great thing about doing testing, you see what works for your particular scenario. But in terms of extracting general rules, it’s tough.

    Most of my focus is on sign up pages for e-mail newsletters and things that work for one don’t always work for another, so while I have quite a few general tips now, I continue to be surprised by the things that don’t necessarily cross over :-)

  2. David Matthews September 18, 2012 at 1:29 pm #

    Great post! I agree with Peter’s comment that occasionally long wins out over short but I like anything that helps people understand and measure the benefits of both focusing and tightening your content.

  3. Andrea from Independent Consultant Journal October 7, 2012 at 10:38 pm #

    Hi – can you point me to more information on how to set up the Google Content Experiments plugin? I have a custom theme and I’m not especially tech-savvy. I’d prefer not to have to involve my web team and other independent consultants/contractors every time I want to play around with a split test. Can I just do this myself in WP? Could my copywriter?

    • Matthew Stibbe October 8, 2012 at 6:51 am #

      It’s pretty straightforward if you’re reasonably familiar with WordPress and you should be able to add it to a custom WordPress theme. The whole process took me about 15 minutes but I’m pretty comfortable in WordPress. I gave the link to the plugin page in the article but in case it wasn’t obvious, here’s the URL: http://wordpress.org/extend/plugins/google-content-experiments/. I guess the acid test is that if the information in this link is mystifying to you, you’ll need to get some help. If not, you should be good on your own. Best of luck, Matthew

Trackbacks/Pingbacks

  1. Why you shouldn't do a website redesign in 2014 | Bad Language - February 4, 2014

    […] to action. This is a huge valuable, reducing the copy on one landing page on Turbine, for example, nearly doubled conversions. The value of A/B testing is that produces measurable improvements without wholesale […]

Leave a Reply