It’s a lot easier to be confident in your ideas when you can say, “We tested it. This messaging outperformed the others. This design increased conversions. This title got the most clicks.”
A/B testing is a practice every marketing team should have in their toolkit. But while the concept is pretty straightforward, how to actually run an A/B test can be tricky. What do you test? How much should you change from one version to the next? How do you know when one version is better?
Here are five tips to help you run successful A/B tests.
1. Test one variable at a time
When you test more than one variable (title, copy, images, types of content, etc.), you can’t prove which variable had which effect. Maybe it was a smart move to remove the testimonial section from your sales page. Maybe it was a bad idea to add all those feature callouts.
If you test multiple variables, you’re measuring the net effect, and you might be mixing positive and negative effects without knowing it.
2. Wait until you achieve statistical significance
When you start A/B testing your content, you might quickly discover a variation that’s vastly outperforming the others. But don’t get hasty. A small amount of data is about as useful as an assumption.
A true A/B test might require thousands of results before it achieves statistical significance.
Statistical significance is when you have enough data to be confident the results did not occur by chance. Usually, statistical significance means you can be 95 percent confident in your result. (And yes, that means there’s a 5 percent chance that the results occurred by chance.)
Optimizely (an enterprise-level A/B testing platform) has a helpful article if you want to know more about statistical significance.
So remember: one variable at a time, and your result has to have statistical significance.
How you actually run an A/B test depends on what you want to test. Here are some of the most common things marketers A/B test, and some of the tools they use to do so.
3. Change one thing at a time on your landing pages
On a landing page, you might test copy, headings, imagery, layout, adding or removing sections, and any number of other elements that contribute to the success of a given page. It’s all too easy to get carried away and test completely different versions of your page.
It takes patience, but focus on one variable at a time. Identify the variables you think are most likely to have an impact, and test those first.
Google Optimize is a simple A/B testing tool that lets you test up to three variations at a time for free. It’s a great option for organizations that aren’t ready for a premium A/B testing tool. But if you’re willing to pay for a more effective website, Unbounce, VWO, and Hotjar have powerful analytical tools to feed more advanced A/B tests. (They all offer free trials, too.)
4. Optimize your emails
If you aren’t already, you should definitely A/B test your subject lines. Otherwise you might be sitting on an idea that could increase open rates by 10 percent or more and you’d never know it! Good email service providers should also give you options to test email content.
For subject lines, here are some things you might test:
- Using punctuation
- Using emojis
- Long subject lines vs. short subject lines
- Asking a question vs. making a statement
- A clear explanation of the email vs. an intriguing hook
In the body of the email, you might test things like:
- Button text
- Button color
- Using headlines
- Images vs. no images
- More copy vs. less copy
- Call to action
Even if your test achieves significance, there can be some variation from email to email. In other words, don’t use one test to establish a universal rule for all future emails. It’s good to test the same variable multiple times. If you continually see the same result, you can use it as a baseline for your ongoing best practices.
The exact method for creating an email A/B test depends on your email service provider. Google “[your email service provider] A/B test” to see how it’ll work for you.
5. Compare your best ideas for blog post titles
After reading Buzzsumo’s list of the most shared headlines, you’re probably ready to test one good headline idea against another. For blog posts, you might want to test titles that evoke different emotional reactions, or that set different tones, or appeal more to different segments of your audience.
Keep in mind: it’s going to take a lot of traffic to achieve statistical significance. If your blog doesn’t get much traffic and you can’t give it a big boost from another marketing channel like email or social, you might want to hold off on these tests (or take the results with a grain of salt).
To test multiple titles on a blog in WordPress, you’ll need a plugin, like Title Experiments Free.
Are you A/B testing yet?
You should be doing this right now. Before you launch another landing page, send another email, or publish another blog post. There are free options to A/B test all three of those things, so there’s no excuse not to do it.
Then, when someone says, “Prove it,” you can say, “OK, I will.” And when someone tries to push through a bad idea, you can say, “Great, let’s test that against our idea.”
