If you take your work seriously and are results-oriented, you don't want to miss out on the enormous potential of A/B Testing.
They can help you uncover high-impact changes to the subject line, design, home pages and more, as detailed in our article "5 aspects of your email marketing to apply A/B Testing to"..
To apply A/B testing to your messages, all you need to do is to present two versions to two small groups of your subscribers and then listen. The metrics will tell you their preferences, and from them you can act in the short term by sending the most efficient version to the rest of your audience, or in the long term by replicating that preference in future campaigns.
It sounds simple and it is even simpler at MasterBase®, thanks to our automation of the process of selecting the most effective part and sending it to the total base. However, it is also quite easy to make mistakes with A/B testing and reach the wrong conclusion, completely undermining the results.
Running a lot of A/B tests only to conclude that you didn't learn anything actionable due to poor execution is painful. To avoid that frustration, we suggest you follow these 6 tips to get the most out of your A/B tests.
-
Focus your A/B testing efforts on the aspects of your campaign that will have the most impact on your results.
Sometimes small adjustments to minor elements can push the needle on your barometer, so focus your testing on the key elements of a message, such as the subject line, calls to action, images and headlines.
But there are also other elements worth trying:
- Different trigger logic for automated messages
- How soon to send the message after it is activated
- To send or not to send a series of automatic messages
- Under what conditions is an automatic message omitted in a series?
-
Limit your A/B testing to one thing at a time.
Unless you are fully engaged in an evaluation process, you should keep your A/B testing limited to one change at a time. For example:
- Green button versus blue button
- Message with social proof included vs. message without social proof
- A lifestyle image versus a product image
- A percentage discount versus a discount in figures.
Modifying more than one factor between versions A and B would make it very difficult to determine which element is responsible for the difference in performance.
-
It starts from a clear hypothesis
Don't make random changes just to see what might work. Focus on what you are looking for and use that as the basis for your testing.
For example, if you are trying to increase conversions, you can create a version of your message where the call to action is above the display or preview (ATL), to make it more visible, and another version where the content precedes the call to action (CTA).
Also, if you are looking to decide whether it is good to address a new subscriber by name in the welcome message subject line, you should test one subject line with personalization and one without.
-
Choose a winning test metric that is aligned with your campaign goal
Most campaigns aim to achieve conversions, so your A/B testing should also be geared towards improving those metrics.
A common misperception is to believe that the subject line can only affect the open rate, that the message can only impact clicks, and that the landing page content can only influence conversions.
This is not true! The different instances do not operate in isolation. Each factor affects the whole, because subscribers experience email marketing campaigns as a whole.
So, the goal of a subject line is not to generate opens, but people who convert. And similarly, the goal of content is not to generate clicks, but people who convert.
-
Develop a periodic testing program and record the results.
The A/B tests designed ad hoc are inefficient because they are sporadic and focus on different objectives. To get the most out of your A/B tests, you need a plan. Develop a testing schedule that tracks:
- The theories you are trying to confirm
- What email addresses are you using to test each theory?
- The results of each test and how they affect your future test plans.
- Try to include an A / B test in at least the 50% of your broadcast.
-
Confirm test results
An individual A/B test is not conclusive forever. In the short term, any lift you see may be the result of the novelty effect. Subscribers are attracted to what's new, which can lead them to overvalue any changes you make.
The novelty effect wears off fairly quickly. Therefore, if you run the same test two or three times over a period of time, that novelty effect will not be present and you will be able to measure the true impact of the change.
Over the long term, consumer behaviors and attitudes vary. The composition of your contact list may also change depending on subscriber acquisition practices, modification of your product or service offerings, expansion into or out of new territories, and other factors.
The more definitive the success of a test, the longer you can wait to reconfirm it. But, over time, you'll want to periodically confirm each test, at least once or twice, which, once again, proves that an A/B testing plan is critical.
