What are some best practices when split testing?
If you’re not A/B testing, or split testing your marketing campaigns, then you’re likely losing out on higher conversion rates, subscribers, and buyers.
Split testing is an essential conversion optimization practice that will help you make the right marketing decisions.
Even billion-dollar companies like Google and Facebook are constantly testing their layout, so why wouldn’t you?
Here’s a pretty decent blog post about 13 common mistakes with split testing and how to avoid them:
13 Dumb A/B Testing Mistakes That Are Wasting Your Time
In this article we’re going to cover some best practices, or “rules” to help you get the best results when split testing.
Before we get into the rules, we’d like to define some terms first so they will make sense to you later on.
First of all, a “control page” is your default page that doesn’t have any changes and will be compared with other pages that do contain changes.
A “page variation” is a page that has certain elements changed, such as a headline or an image, so that it can be compared with the main control page.
Finally, if we say that a test has “statistical significance”, that means that we are 95% confident that the result wasn’t attributed to pure chance.
Now that you know some basic definitions, let’s get into the rules!
Table of Contents
Rule #1: Test the Big Things First
There are many things that you could split test: the color of your button, the size of your headline, the word at the bottom of your page, and so on…
But these are very small changes that won’t make much of an impact to your overall conversion rates.
For starters, here are the six things that you want to split test first in no particular order:
- The headline
- Copy text
- Button copy and other call-to-action text
- Social proof
The real key here is that you want to test these six items in radically different ways, for example an entirely different headline, a completely different image, and so on..
Don’t just change a word or two in your headline – change the entire message and see what the results will be. The same idea applies to the other items on the list, such as your images and social proof.
Sure, changing the color of your button might increase your conversion rates by 0.5% after testing for a month …
But is it really worth spending a month split testing the button when you could have been testing the headline and seen a 50% increase in conversion rates instead?
The other issue with split testing small items is that it will require way more traffic and therefore take a lot more time to reach statistically significant results (results that aren’t attributed to pure chance).
As a result, some of these small changes never reach a high level of significance, rendering the entire test useless.
So just remember to test the big things first before moving onto the smaller items.
Rule #2: Test Only One Thing at a Time
Split testing is fun, we get it.
You want to test your headline, your images, and the text copy all at the same time…
But if you test all these things at once and see an increase or decrease in conversion rates, you won’t know which factor caused that change…
So keep it simple and split test one thing at a time.
First test the headline and get the results. Then test the images and get the results, and so on.
The main takeaway here is to make sure that you test only ONE thing at a time.
Rule #3: Don’t Make Any Changes Until the Test Is Done
You might get really excited and end a test early if you see a variation is underperforming or outperforming the control page.
But you should never end a test early.
Do not trust your gut when it comes to split testing — your results will NOT be reliable and the whole split test becomes meaningless...
Instead, wait until your results are statistically significant. With ClickMagick, you
don’t have to worry about this since we
automatically do the math for you and email you when we determine a statistically valid winner.
Depending on how much traffic you’re sending, determining a winner can take minutes, hours, days, weeks, or even months. But be patient and don’t make any changes until the results are in.
Rule #4: Test One Variation at at Time
The more variations you have, the longer it will take to get statistically valid results.
For example, if you are split testing two page variations and have purchased 1,000 clicks, that gives about 500 clicks per page variation.
But if you’re split testing 10 variations against each other, you’re now splitting that same 1,000 clicks between those 10 pages, getting about 100 clicks per page.
100 clicks doesn’t provide the same useful information that 500 clicks would.
The only reason why you would want to test more than one variation at one time is if the variations are closely related and tied to each other, such as if you have different pricing plans and want to test them against each other.
But for other items, such as ad copy or images, testing only one variation will almost always yield statistically significant results faster.
So make sure to test one variation at a time.
Rule #5: Keep on Testing Forever and Ever
No matter how amazing you are at marketing, you should never stop testing.
All the top marketers in the world like Neil Patel, Russell Brunson, Todd Brown are constantly split testing their campaigns.
Even Google and Facebook, the richest companies in the world still split test their pages. You can see this yourself when you log into their ad platforms – the layout changes almost every day.
So keep on testing because no matter how good you think you are, there are just some things that will always come to you as a surprise when you split test.
It might also be a good idea to follow some of the top marketers and pick up some split testing pointers from them.
For example, here is an excellent blog post from Neil Patel talking about split testing:
How to Master A/B Split Testing Quickly (And Increase Conversion Rate
So that’s about it for our quick “rules”.
Ready to start split testing yourself?
Take a look at this article and get started right away:
How do I set up a split test?
Article 468 Last updated: 07/02/2020 10:54:17 AM