Sometimes, as marketers, we get so caught up in what we think looks great that we forget to think about what our audience prefers.
Thus, we launch what seems like a winning email campaign only to fail.
You spent hours designing the perfect template, the perfect CTA button, and the perfect image.
Yet, for some reason, people are not interested in your email.
You can use A/B testing to pinpoint what’s working (and what’s not).
In this blog post, we’ll show you:
- What email A/B testing is
- How email A/B testing works
- Why email split testing matters and the benefits
- Eleven fantastic tips to boost your email open rates through A/B testing.
Let’s get started!
What Is A/B Testing?
A/B testing compares two versions of a webpage or applications against each other by tweaking a single element to determine which one has the better performance.
In essence, you have version A and version B — and you tweak one element of your email marketing campaign at a time.
Keep in mind that just because something is newer doesn’t always mean it’s improved. You have to be willing to listen to the results and let them drive your decision.
Okay, so what should you split test? Well, that’s mainly up to you. Perhaps you want to pit two versions of your homepage against one another. You can also test your landing pages or the look of an app.
Sometimes, it’s not about testing the entire page but certain elements.
For instance, does font A or font B look better? Should you go with a bright color scheme or one that’s darker and more muted? Does the CTA button drive more results at the top of the screen or in the center? Should it be blue or pink? What about the wording?
Here’s an amazing video that explains the science behind A/B testing:
It’s always better to tweak a single element at a time to gauge the performance difference — otherwise, how can you pinpoint which change performed better?
How Email A/B Testing Works
If you’re not already using A/B or split testing, you should start now.
One roadblock that may hold you back is your lack of understanding of how split testing works.
It sounds so easy, but what kinds of tools and software do you use to test the control against the variation? Who decides which one is the winning variant?
Those are all perfectly valid questions.
In this section, we’ll take a basic example, two versions of an email, and use split testing to determine which would have the best results.
Here’s how to begin a test email campaign with four popular email platforms: EngageBay, Campaign Monitor, AWeber, and MailChimp.
The all-in-one sales and marketing platform EngageBay is an excellent resource for split-testing.
You can test the headlines within that feature to determine which one would generate the highest open rate.
#2. Campaign Monitor
If you use Campaign Monitor for your email marketing, then you’re one of about 250,000 people and counting.
Campaign Monitor has its A/B testing in email marketing built right in. You can compare content such as the email body, the from name, and the subject line.
Pick one to start with.
You send between 20% and 30% of your overall email list both versions of the email.
You even get to select the conditions that determine the winner: people who click on the link, the total unique clicks, or the open rate. You get to pick the length of the test as well.
Aweber is another excellent email marketing software with A/B testing tools.
You will need a contact list of at least 100 people for AWeber’s most accurate split testing results, so make sure your list is up to par.
To begin testing, choose the Broadcast Split Tests option. Make sure you only use two variants instead of four.
Select how many contacts you want to receive the message as a percentage. Then you can use AWeber’s HTML coding, plain text messaging, or drag-and-drop email builder.
From there, pick the subject line or another part of your email you want to test. It’s your job to make both emails identical save for the element you’re split testing. Once your emails go live, AWeber sends you the results.
This tells you who wins. Using AWeber for A/B testing is somewhat more complicated than Campaign Monitor, so keep that in mind.
You can also use MailChimp for your split testing needs.
They suggest using 20 to 50 percent of your audience for split testing purposes. Then, you must choose what the winning metric is, such as higher open rates, click rates, or views. You again get to determine how long the testing will run.
To wrap up, include the testing information in the form fields. Then your emails go out. You can do this yourself or let MailChimp automate the email sending. Once you know how your recipients respond to your campaign, you can use the test result for your future email.
The Benefits of A/B Testing Your Email Campaign
How hard does your small business currently work to obtain more customers? The money you’d spend on A/B testing is a fraction of your current budget, yet split-testing can be an assured method for increasing open and click-through rates.
Another perk of A/B testing: It tells you what works so you can do it again. Sometimes, getting an email campaign right feels like there’s a bit of luck and magic involved.
Not so with split-testing. You’ll know whether your newly-positioned CTA worked or if it was that awesome subject line. You can then replicate these tactics in future campaigns.
Why Does Email A/B Testing Matter?
Okay, but A/B testing can be so time-consuming. Can’t you just skip it? Why does it matter anyway?
A/B testing matters because email marketing does. It’s as simple as that.
Sending emails to your audience that go unopened or get deleted is a waste of your company’s time, effort, and manpower. Oh, and most of all, it’s a waste of money.
As a small business, every penny counts. You want to ensure your efforts will yield profitable returns. That’s why you cannot skip A/B testing.
Email A/B Testing Rules to Follow
For the success of your split testing, make sure you follow these rules and best practices:
- Depending on the type of company you run, you’ll focus on different testing metrics. CTAs, homepage messaging, free trial signup flow, and lead form components matter a lot for tech businesses.
- If you’re in e-commerce, pay special attention to testing your funnel components, navigation elements, and homepage promotions. This helps you boost your sales over the holidays and the average order volume and checkouts.
- For travel businesses, A/B test for such metrics as ancillary product presentations, search results pages, and homepage searches.
- Media companies strive to boost their social sharing while growing their readership. They also want to maintain the readers they have, keeping them on the site longer. It’s important to run split tests on social sharing buttons, recommended content, and email sign-ups to meet your goals.
- No matter what industry you’re in, when you’re done testing, make sure you remove remnants of the test. The longer these remain on your site, the better the chances of the Google algorithm picking upon them. This can appear deceptive to both Google and other search engines. When your test ends, leave no traces.
- On a similar note, make sure you’re not testing for a very long time, either. For instance, there’s very rarely any need to test for more than 24 hours. You’re not reaching out to your entire contact list. Those who don’t respond won’t, but you’ll still get good enough results if most people participate.
- To make it easier to get rid of your split testing pages, make them 302 redirects. These are temporary by design. If Google or another search engine picks up on these through their algorithm, they know these pages won’t stick around forever. They also won’t index this temporary page. Compare that to a 301 redirect, which is permanent and can thus get indexed. If both split test pages are set as 301 redirects, this can hurt you.
- The rel = “canonical” attribute can also help to this end, especially if your test has more than one website URL. This attribute tells Googlebot that all the URL variations should go back to your main page, which was there originally. This also prevents the algorithm from dinging you.
- You shouldn’t cloak either, which is when you limit the content to a search engine. The customer or test participant can see the full range of content, but a search engine cannot. Limit the segmentation you do by IP address or user agent address, so you’re not at risk of cloaking. If you’re caught doing it, you could face the consequences. Your page could disappear on any results pages through Google or other search engines.
11 Tips to Improve Your Campaign’s Open Rates Through Email A/B Testing
Now for the main course: Here are 11 tips to improve your campaign’s open rates through email A/B testing.
You don’t have to apply everything we recommend — mix and match to suit your needs.
1. Link Count Testing
How many links are in your emails? Is it just one or two, or do you have way more than that?
While we recommend you test this, the fewer links you have, the better. There are two reasons for this.
The first is a psychological one that employs what’s known as the paradox of choice.
Perhaps you’ve heard of the famous jam study. If not, here’s a recap: two groups of people were at a supermarket. Both got a selection of jams to pick from, one with six flavors and the other with 24 flavors.
While people liked the table with more varieties of jam, they didn’t buy nearly as much jam as those with fewer options. That’s because the more choices we get, the harder it becomes to pick something out. In the end, most people will just walk out empty-handed.
The same thing applies to your links. The more you have, the more challenging it becomes for the email recipient to decide which ones to click on. In the end, they may not click on any of them.
The second reason we recommend you cut down on links is for aesthetic reasons. It’s because too many links look spammy. Your email could get stuck in the spam filter instead of the recipient’s inbox.
2. Email Send Day/Time Testing
We mentioned this numerous times in our blogs: Tuesday, Wednesday, and Thursday are your best days to send emails. The ideal times to send emails are early in the morning, midafternoon, and later in the evening.
Okay, so if that’s all so concrete, why bother with email marketing testing? Well, because those times and days can fluctuate slightly depending on your audience.
Even if you think you know the best days and times your audience is most receptive to your emails, it could change. This will always be in flux, and you have to be prepared accordingly.
3. Email CTA Testing
Ah, the call to action. It’s so necessary, but is it working for your company?
Your email CTA can appear in several formats. Perhaps it’s a line of text at the end of your emails. Maybe it’s the centerpiece of your email body. There’s proven research that shows the CTA button is more effective than a link in emails. According to data from Campaign Monitor, it’s possible to see a 28% boost in your click-through rate with a button.
That might be the first thing you test: buttons or links?
Once you make that decision, you’re not done yet. You should also test for the following:
- The amount of whitespace your email has, which can either obscure your CTA button or let it stand out
- The color of your CTA button, as well as the other colors in your email, including the background
- The design of the button, including shadows, edges, and more
- The size of the button
- The placement of the button
- The CTA button copy
4. Anchor Text Testing
Are you using anchor text in your emails? If not, you’re missing out on a great way to pique the interest of your audience.
What exactly is anchor text, anyway?
This is a hyperlinked copy that often ends in an ellipsis. It’s a preview of what’s to come and draws out their natural sense of curiosity. Here’s an image that showcases how anchor text can play to several emotions.
The first option you have is to get your audience curious. If this is the approach you take, then the accomplishment you highlight should be something they want to do. Make sure you don’t give the whole secret away in your anchor text.
You can also invoke empathy in your audience, but you’d have to know their pain points to do this.
Maybe your audience is having a tough time converting leads or getting anyone to read their blog. By showing you’ve had the same problems and overcame them, you display your empathy.
Finally, you can invoke a sense of fear by making them feel like they’re missing out on something extraordinary. That’s what FOMO is all about, after all.
No matter which of the emotions you go for in your anchor text (and maybe it’s another emotion entirely), remember this is just a preview. Once the reader clicks the link, they’ll get the whole story.
5. Personalization Testing
You know you should personalize your emails, but how much personalization is enough? Only A/B testing can answer that question. Here’s a great example of a travel company. You can see two personalized emails they tested side by side.
In the first version of the email, it isn’t super personalized. Yes, it says “featured hotels just for you,” but that’s about as far as it goes.
On the other side of the email are all the exotic locations a customer might want to travel to. It’s a good email, but is it great?
Not as great as the second email. This one has the name of the email recipient outlined in the sand. Their name is also in the body’s first line beneath the image.
How excited would you be to see your name traced in the sand? Very, right? That alone is going to make you want to read through the email.
Get creative with your personalization — and then A/B test it.
6. Tone Testing
How do you write your emails?
Perhaps you take on a more informal tone, such as the voice of this blog post. It’s relaxed yet informative and feels like talking to someone you know. You’re reaching out to your email audience on a more personal level.
Maybe you prefer the formal approach. The writing is more buttoned-up and may include jargon and terms that only those in your niche would understand.
Both are appropriate in some instances and industries.
Also, your tone might vary depending on which audience segment you’re trying to reach. You might write in a more relaxed tone with your established customers and more formally with your new ones.
The opposite could also be true.
Split testing is the best way to know what your audience likes best.
Prepare two versions of the same email, one with an informal tone and the other written formally. Then see which one gets the highest open rate.
Testing for email tone is not a one-and-done thing. You’ll have to keep repeating the test when sending major emails to new and established customers.
7. Content-Length Testing
No one likes slogging through a never-ending copy of an email. Going in the opposite direction can be risky, too, since brevity can also be vague if you’re not careful.
We recommend testing at least three versions of an email. One should be the long one you prepared with love and time.
The other should be a mid-length version that’s shorter but not too short. Finally, condense your message into the lowest number of words possible.
Now test all three emails to see which gets the highest open rates. We’d bet it’s probably the mid-length one. That’s not to say that longer emails never have their place. For example, in a newsletter email, your audience expects a longer, meatier message.
Short emails are also appropriate in some instances. If you’re sending a reminder about an upcoming event or webinar, short messages are more appropriate.
Are you having a hard time cutting your emails down to size? You could always try adding some images to break up longer blocks of text.
A study by HubSpot found that many consumers (65 percent) liked seeing images in emails more than they did text.
Have you written about the subject before in a blog post? Then link to that rather than devoting paragraphs to the same topic in your email.
This lets the audience choose if they want to read further. Just remember not to add too many links.
8. Link Text Length Testing
Speaking of links, there are so many ways you can format them.
For instance, let’s say we’re linking to our EngageBay homepage. We could just say “check out EngageBay for all your marketing need” and link to the company’s name. Alternately, we could do something like this: “if you need an all-in-one sales and marketing software.”
When you add hyperlinks to your email, how should you do it? How many words should be included in the links? This is something else you’ll have to test.
Most marketers agree that 12 words are the maximum amount. Here’s a MailChimp illustration that shows that data in more depth.
It appears the more words you cram into your email links, the lower and lower the email open rate is.
9. Greeting Testing
Did you know the very opening words of your email can be a big determinant of your campaign’s open rate success? It’s true.
Whether you say “hello,” “howdy,” or “good morning” does matter. Use A/B testing to determine what types of openings influence your readers the most.
Like the body of the email, the tone of your email greeting will vary.
If this is a new customer and you are keeping a somewhat formal tone, you might write “welcome” as your greeter. Starting your message with a “hey there” is perfectly appropriate for older, established customers.
What seems to matter most in terms of open rates is greeting personalization. Instead of “Hello,” try “Hello, [name].”
10. Subject Line Copy Testing
One of the most important parts of any email campaign is the subject line. After all, this is the first impression you make on your email recipient.
It has to be a good one.
There are lots of ways to craft a fantastic subject line.
Personalization is always good. Stats and figures can pique interest. Discounts and deals get most people clicking.
Your company can even use emojis — but don’t overdo it.
Whatever works for you, you have to know for sure. An A/B test will truly come in handy here.
Like your anchor text, your subject line can connect with your audience on an emotional level. You can use empathetic lines to connect with your readers.
You can also foster that sense of FOMO by sending an email about a deal that’s only good for today or a sale that’ll last 12 hours more.
You can get your audience curious by writing a subject line much like you would your anchor text — by making a partial statement, ending it with an ellipsis. By telling only part of the story, you hopefully intrigue your reader enough that they want to click, open the email, and see how the ending goes.
11. Subject Line Length Testing
Okay, so you’re confident the content of your subject line will appeal to your audience. You A/B tested it and everything. You’re all ready to send the email, right?
Not so fast.
You should also test for the length of your subject line before you launch your campaign.
Exceeding 50 characters is no good, as there’s a chance the audience member might see a cut-off version of your subject line. This depends on the device they’re using, of course.
Your audience is more likely to see the crudely chopped subject line on smartphones.
Five words are the sweet spot. This should fit the character count requirement so that even if your audience is on their phone and holding it in portrait mode, they should see the whole subject line.
A/B testing in email marketing is crucial for your campaign’s success. It will influence email open rates and click-through rates as well.
By starting an A/B testing campaign, you can be sure all elements of that campaign resonate with your audience.
Nothing is too small to split-test. From the placement of your CTA button to which time to send an email, it’s all worth it. Some elements of your campaign will have to be tested regularly and others just once or twice.
Knowing your audience allows you to create stellar, winning campaigns. A/B testing ensures all parts of your campaign are tailored to their interests and pain points, winning you loyal, buying customers.
The great thing is you don’t need to do everything manually. For small businesses, EngageBay is an excellent option. It lets you A/B test your email campaigns by letting you create a variant and then test it against your main campaign. You can choose the winner, and voila: your open rates go through the roof!
Sign up now, and A/B test your way to success.