Ashley Hill — October 27, 2019 
Have you considered A/B testing yet? It’s a great way to determine how your visitors behave and how you can best market to their needs and improve their user experience while boosting your conversion rate, click-through rate, or any other element of your website or your email marketing campaign that you want to develop.
A/B Testing is a great way to figure out what works best for your company, your visitors, and your marketing campaigns.
It’s a marketing experiment where you split your audience and provide each group with a different version of the same content to see which performs the best – and which your visitors respond to more. You show Group A one version of a piece of marketing, and Group B a different version, and test the response to each.
But A/B testing can be tricky. If you aren’t careful about how you conduct an A/B test, you may end up making wrong assumptions about how your audience reacts to a certain marketing technique. So here’s what you need to know about A/B testing, and how to conquer it.
1. You Should Only Test One Change at a Time
Just like if you were conducting an experiment back in your high school biology class, you want to stick to the scientific method. And part of the scientific method is having a control group and a variable group.
You only want to test one change at a time so that you can accurately assess which test group had a high success rate – whether that is a higher click-through rate, a higher conversion rate, a lower bounce rate, more data collected, or whatever it is you are hoping to achieve with your test.
So what kinds of things can you switch out in an A/B test? The answer? Truly anything you like! Here are a few ideas to get you started:
- The text OR visuals of a call-to-action button
- The headline or subject line
- The main graphic or picture
- The actual sales copy or product description
- Landing page graphics or copy
This is true for your website, or an email marketing campaign. You can also test the effectiveness of a particular ad, like a sponsored search result for Google Ads, and although there will be fewer options to change, you can still test how well one campaign does over the other.
2. Testing Takes Time
You don’t want to rush the process of A/B testing. Allow the test time to work. A/B testing is not an overnight project, so depending on how much traffic your website gets, you may want to give the test a few days to a few weeks to gather enough data to get a clear picture of your visitors’ preferences.
If you’re testing an email campaign, this process will be shorter – probably no more than a few days – but you still have to give the test time to work, and for recipients to check their inbox.
Hubspot has a great article about determining the length of time for an email marketing A/B test. Their suggestion? Check data you already have.
If 70% of your emails are opened within the first 24 hours, and then it tapers off to 5% opened in the following days, chances are you can limit your test length to 24 hours and still get great data. For A/B testing of a website though? You definitely need to allow for more time so you can obtain a larger test group – the visitors to your website.
3. Have a Goal in Mind – and Remain Focused on That
When you’re establishing an A/B test, determine what it is you are trying to accomplish. Maybe that’s a higher conversion rate from a landing page? If so, don’t be swayed by other data; keep your eye fixed on your original goal.
Let’s say you start an A/B test looking to improve your landing page’s conversion rate. While Test Group A has a much higher click-through rate, Test Group B ends up with a lesser click-through rate, but within that group, there’s a much higher ratio of conversion, comparatively. Sure, Group A had a high click-through rate but did not convert as many visitors.
It may be tempting to implement the Group A principle on your landing page. Remember though, in this example, you were seeking a higher rate of conversion. Stick with your original goal, and remain focused on the results you are seeking.
4. Your Results Need to Be Statistically Significant Before You Make Changes
Did your variable group perform a lot better than the control group, or just slightly better? Are the results of your test major enough to warrant making a change? If not, that’s okay. That just means that your variable didn’t impact results.
How do you figure this out? Simply divide the number of successful goals completed (new leads, conversions, email signups, etc) by the number of tries (emails sent, page visits, impressions seen, etc) to find the percentage of your success rate.
Chances are the data will immediately be very telling, and you can begin to carefully and thoughtfully implement some changes. And if not? You can try some new variables in a new round of testing.
5. It’s Perfectly Okay to Ask Real Customers for Feedback
Want more insight into why something on your page is or isn’t working? You may want to consider an exit poll: Just a question or two to find out why your visitors are behaving the way they are. A/B testing provides you with solid, reliable quantitative data, but what it doesn’t provide you with? Your visitors’ rationales that explain their behavior.
Sites like Survey Monkey can create exit surveys to gather this information during your testing period.
A/B testing is a great time to add a small exit poll to explain customer behavior. Maybe there’s a high click-through rate to download your e-book, but when visitors see the cost, they change their mind. This provides you with even more data to determine how you can best serve your customers and how to convert the most visitors into customers.
6. You Need a Large List of Contacts for a Sample Email Marketing A/B Test
Ideally, you are conducting an A/B test to find out what your customers respond to most positively. If you plan to test a certain kind of email marketing campaign, you need to have a long contact list. Here’s why:
When you send out A and B versions of a marketing campaign, you are truly testing to figure out which will be more successful. You send out two versions of the same marketing email to a portion of your contact list, determine the results, and then you can send an email with the better response to the rest of your contact list.
You need a long contact list so you can send your test to a large enough sample group to get statistically significant results first, and still send your “final” email after testing.
Can you send out two versions of your email to your entire contact list, and gather the data to improve future marketing campaigns? Sure. In fact, if your contact list is less than 1,000 contacts, most email marketing platforms will want you to do so.
But the more you can cultivate your contact list? The bigger your audience will be, the more people will hear your message, and the more effective your A/B testing will be.
If you haven’t already, now is the time to dip your toe into the A/B testing pool – or just jump right in. It’s the best way to learn what really works for your clients, contacts, and visitors. And it will save you marketing money and get you a better marketing ROI in the long run.
Digital & Social Articles on Business 2 Community
(35)
