Do you know what split testing is? Sometimes it’s called A/B testing. When applied to your website, the goal of split testing is to identify changes that increase a desired outcome.
What that means is that you can be looking at one of your web pages and you can ask yourself a question, “Hmm, I wonder if more people would call us if we moved the phone number to the upper right?”
But instead of just wondering, you run a test to find out the answer. You test the hypothesis to see if there is a statistically significant change.
Effectively, what you do, is you create an alternate version of your web page. You create a version that fulfills your own question. In this example, you would create a page that has the phone number moved to the upper right. At the same time you continue to keep your current page.
Now you split test them. Each person who visits your website is given one of the two pages randomly. You don’t know which one they are getting. And they don’t know that they are involved in a test.
Since this example is about phone calls from your website, the person answering your phone has to complete the measurement. For example, “Thanks for calling. Just for our records are you seeing the phone number next to Jack our president or in the upper right of web page?” Tally the answers.
The more answers you have, the more accurate the data will be, so collect the data for at least a couple weeks unless the phone is ringing off the hook!
You are collecting 4 pieces of information. The number of times someone saw the original page, the number of times someone saw the test page, the number of times someone called from the original page, and the number of times someone called from the test page.
Let’s say your original web page was seen 412 times during the experiment and the new page was seen 421 times (the pages are displayed randomly and should each be close to each other but will vary). You got 49 calls from the original page and 58 calls from the second.
Drop those 4 numbers into a handy split testing calculator and you will see where you stand with this test.
In this case, the conversion rate for Sample B, your brilliant idea, is a little bit better than the original.
The confidence is much better, nearly 80%. At this point you might be tempted to update your website to go with the new page.
But first you must understand what the confidence number means. It tells you that nearly 80% of the time, this page is the better page.
What that means is that 20% of the time, or 1 out of 5 times, changing to this page will be a mistake.
The general rule of thumb is that you want to see a confidence of 95% or higher before you commit to a change. In that case, your data is misguiding you only 1 out of 20 times. Most of the time you can live with that.
Of course, you get to decide your level of comfort with the confidence numbers. If you think being wrong 1 out of 5 times is acceptable, then go ahead and implement the change. I would suggest considering other factors first, however.
If you collected all this data within days or even a week or two, let the test run a little longer. If the ratios stay the same the confidence goes up. If confidence goes down then you might be happy you waited.
On the other hand, if it took you a year to get this much data, then you have to work on getting more traffic to run through your test!
The upside of split testing, is that every time you make a change that improves conversion, you are creating a better website for your business. You are getting better results with more leads and more sales. You can split test your way to bigger profits…
Here’s the link to the split test calculator I used: www.splittesttool.com. Try it yourself.
