Knowledge Base      

How do I analyze my Split Test results?

To view the results of your split test, go back to the Tools menu and choose “Split Testing” again.

This will show you the stats for your control page and all of your split test variations, including the number of clicks, conversions and the conversion rates for each.

In addition to these basic stats for each page, ClickMagick will also show you how the conversions for each variation compare to your control page — represented by a percentage difference.

For example, if a variation has a Difference value of +78% highlighted in green, this means that particular variation is doing 78% better than your control page. On the other hand, a Difference of -42% highlighted in red means that your variation is doing 42% worse than your control page.

Even better, ClickMagick will also do all the complicated math to determine when you have a statistically valid winner.

If you tell ClickMagick that you want to optimize for Sales for example, once you have a statistically valid winner based on Sales ClickMagick will automatically end the Split Test and start routing all of your traffic to the winner in order to maximize the value of every click. You’ll also receive an alert via email.

If you do not want ClickMagick to do this, you can run what we call a Manual Test in which case ClickMagick will simply show you the results and you can end the test manually whenever you want.

Just keep in mind that until ClickMagick shows that you have a statistically valid winner or loser the results are not statistically valid and you should not make any changes based on these inconclusive results.

And just so you know, ClickMagick considers a result to be statistically valid when the “confidence interval” is 95% or higher, which is the generally accepted value to use for this purpose.
Note: Due to random chance, a page could theoretically get 10 clicks and 8 sales, even if your average conversion rate is only 3%. This type of thing is very rare, but to prevent this from ruining your split test ClickMagick waits until your Primary URL and any variations have at least 100 clicks before attempting to determine a winner.

What if I never get a statistically valid winner?

If ClickMagick never declares a winner because none of your variations rises above a confidence of 95%, that simply means there is no winner and no action should be taken. Think of it as a “tie.”

The biggest mistake that people make when split testing is guessing at the results or making changes to their website or funnel without statistically valid data to inform those decisions.

Do not do this under any circumstances or you’re essentially throwing money out the window. Even marketers who’ve been doing this for 20 years can’t guess which page will win a split test.

If you’re regularly running split tests without any clear winner, it simply means your variations are too similar and are therefore inherently much less likely to be statistically different from each other.

You may see marketers talking about how they got a 40% increase in opt-ins by changing the color of the submit button or other silly things like that – but don’t believe it.

99% of the time changing small things like button colors or a few words here and there are not going to result in statistically valid differences. And when it comes to split testing most marketers, even “gurus”, have no clue what they’re doing and no idea whether their data is statistically significant or not.

When split testing the idea is to test “big” changes – a completely different headline, a different offer, higher or lower pricing, more testimonials, etc. Then do it over and over and over again.
Note: If you want to end a split test before a winner is chosen, simply delete all the split test URLs by clicking on the Delete icon next to each link.

Article 519 Last updated: 06/04/2021 5:41:11 AM