Improve Email Campaign Open Rates Through A/B Testing

Sometimes, as marketers, we get so caught up in what we think looks great that we forget to think about what our audience would like.

Thus, we launch what seems like a winning email campaign only for it to fail. That’s why, before you ever hit send, you have to make sure you do email A/B testing.

Also, known as split testing, A/B testing is a crucial part of a marketer’s job.

With this testing, you take two variants, A and B. Then you test among your audience or similar audiences to gauge which one is preferable.

An A/B testing campaign can be the difference between awesome open rates and abysmal ones. That’s why it’s so important you make this testing a part of every email you send.

If you’re unfamiliar with A/B testing but want to start, we recommend you begin with this article.

In it, we’ll share 11 email testing ideas you can use for your own campaigns. Each of them should increase your open rates.

What Is A/B Testing?

Before we can dive into the many ways you can see more email campaign success through A/B testing, we have to make sure you have a clear understanding of what this type of testing even is.

Referred to as bucket testing or split testing, A/B testing involves looking at two versions of the same thing to see which one is better.

So, you’d take version A of something and then version B, hence the name A/B testing.

Image courtesy of Optimizely

The A in your testing is the control while the B is the variation. You can also think of A as the original or older version and B as the newer version.

Do keep in mind that just because something is newer doesn’t always mean it’s improved.

You have to be willing to listen to the results and let them drive your decision. That sometimes means choosing the version of a page or design you don’t like as much, but it’s not your choice at the end of the day.

Okay, so what should you split test? Well, that’s largely up to you. Perhaps you want to pit two versions of your homepage against one another. You can also test your landing pages or the look of an app.

Sometimes, it’s not about testing the entire page, but certain elements.

For instance, does font A or font B look better? Should you go with a bright color scheme or one that’s darker and more muted? Does the CTA button drive more results at the top of the screen or in the center? Should it be blue or pink? What about the wording?

You can answer all those questions with split testing, making it an incredibly useful part of any marketer’s arsenal.

How Email A/B Testing Works

If you’re not already using A/B or split testing, then you’re going to want to change that.

One roadblock that may hold you back is your lack of understanding of how split testing works.

It sounds so easy, but what kinds of tools and software do you use to test the control against the variation? Who decides which version reigns supreme?

Those are all perfectly valid questions. In this section, we’ll take a basic example, two versions of an email, and use split testing to determine which would have the best results.

We’ll also go step by step and explain how you would do this form of testing yourself. You can then apply this information and perform your own split tests.

Here’s how to begin a test email campaign with four popular email platforms: EngageBay, Campaign Monitor, AWeber, and MailChimp.


The all-in-one sales and marketing platform EngageBay is a great resource for split-testing. Here’s an informational video on YouTube you can watch to acquaint yourself with the platform.

EngageBay’s A/B testing works as follows. You write several versions of your email subject line, then use the Email A/B Testing Feature.

Within that feature, you can test the headlines to determine which one would generate the highest open rate.

Reduce Bounce Rates With Email A/B Testing

Campaign Monitor

If you use Campaign Monitor for your email marketing, then you’re one of about 250,000 people and counting.

Campaign Monitor has its own A/B testing in email marketing built right in. You can compare content such as the email body, the from name, and the subject line. Pick one to start with.

You send between 20 and 30 percent of your overall email list both versions of the email. You even get to select the conditions that determine the winner.

For instance, most people who click on the link, the total unique clicks, or the open rate. You get to pick the length of the test as well, such as seven hours, more time, or less.

Image courtesy of Zapier


What if you prefer AWeber instead? After all, this app integrates with Facebook, PayPal, WordPress, and more, allowing you to add contacts from those sources.

You will need a contact list of at least 100 people for the most accurate split testing results from AWeber, so make sure your list is up to par.

To begin testing, choose the Broadcast Split Tests option. Make sure you only use two variants instead of four, as that makes split testing much more complicated.

Select how many contacts you want to receive the message as a percentage. Then you can use AWeber’s HTML coding, plain text messaging, or drag and drop email builder.

From there, pick the subject line or another part of your email you want to test. It’s your job to make both emails identical save for the element you’re split testing. Once your emails go live, AWeber sends you the results.

Image courtesy of Zapier

This tells you who wins. Using AWeber for A/B testing is somewhat more complicated than Campaign Monitor, so do keep that in mind.


You can also use the tried and true MailChimp for your split testing needs. As one of the biggest names in email, their A/B Split Campaign is easy to navigate.

They suggest using 20 to 50 percent of your audience for split testing purposes. Then, it’s your duty to choose what the winning metric is, such as higher open rates, click rates, or views. You again get to determine how long the testing will run.

mailchimp ab testing

To wrap up, include the testing information in the form fields. Then your emails go out. You can do this yourself at your leisure or let MailChimp automate the email sending.

The Benefits of Split-Testing Your Email Campaign

How hard does your small business currently work to obtain more customers? The money you’d spend on A/B testing is a fraction of your current budget, yet split-testing can be an assured method for increasing open and click-through rates.

Another perk of A/B testing is it tells you what works so you can do it again. Sometimes, getting an email campaign right feels like there’s a bit of luck and magic involved.

Not so with split-testing. You’ll know whether your newly-positioned CTA worked or if it was that awesome subject line. You can then replicate these tactics in future campaigns.

Why Does Email A/B Testing Matter?

Okay, but A/B testing can be so time-consuming. Can’t you just skip it? Why does it matter anyway?

A/B testing matters because email marketing does. It’s as simple as that.

Sending emails to your audience that go unopened or get deleted is a waste of your company’s time, effort, and manpower. Oh, and most of all, it’s a waste of money.

As a small business, every penny counts. You want to ensure that when and where you spend will yield fruitful returns. That’s why you cannot skip A/B testing.

A/B Testing Rules to Follow

For the success of your split testing, make sure you follow these rules and best practices:

  • Depending on the type of company you run, you’ll focus on different testing metrics. CTAs, homepage messaging, free trial signup flow and lead form components matter a lot for tech businesses, for example.
  • If you’re in e-commerce, then pay special attention to testing your checkout funnel components, navigation elements, and homepage promotions especially. This helps you boost your sales over the holidays as well as the average order volume and checkouts buyers complete.
  • For those travel businesses, you want more revenue, of course, as well as completed bookings. You should A/B test for such metrics as ancillary product presentations, search results pages, and homepage searches.
  • Media companies strive to boost their social sharing while growing their readership. They also want to maintain the readers they do have, keeping them on the site longer. It’s important to run split tests on social sharing buttons, recommended content, and email sign-ups to meet your goals.
  • No matter what industry you’re in, when you’re done testing, make sure you remove remnants of the test. The longer these remain on your site, the better the chances of the Google algorithm picking up on it. This can appear as deceptive to both Google and other search engines. When your test ends, leave no traces.
  • On a similar note, make sure you’re not testing for a very long time, either. There’s very rarely any need to test for more than 24 hours, for instance. You’re not reaching out to your entire contact list. Those who don’t respond won’t, but you’ll still get good enough results if most people participate.
  • To make it easier to get rid of your split testing pages, make them 302 redirects. These are temporary by design. If Google or another search engine picks up on these through their algorithm, then they know these pages won’t stick around forever. They also won’t index this temporary page. Compare that to a 301 redirect, which is permanent and can thus get indexed. If both split test pages are set as 301 redirects, this can hurt you, as we said.
  • The rel=“canonical” attribute can also help to this end, especially if your test has more than one website URL. This attribute tells Googlebot that all the URL variations should go back to your main page, the one that was there originally. This also prevents the algorithm from dinging you.
  • You shouldn’t cloak either, which is when you limit the content to a search engine. The customer or test participant can see the full range of content, but a search engine cannot. Limit the segmentation you do by IP address or user agent address so you’re not at risk of cloaking. If you’re caught doing it, you could face nasty consequences. Your page could disappear on any results pages through Google or other search engines. In some cases, you only get demoted, but even that wouldn’t be great.

11 Tips to Improve Your Email Campaign’s Open Rates Through A/B Testing

Now, without further ado, here are 11 methods for using A/B testing to boost the open rates of your email campaigns.

1. Link Count Testing

How many links are in your emails? Is it just one or two or do you have way more than that, like five or six?

While we recommend you test this, generally, the fewer links you have, the better. There are two reasons for this.

The first is a psychological one that employs what’s known as the paradox of choice.

Perhaps you’ve heard of the famous jam study. If not, here’s a recap: two groups of people were at a supermarket. Both got a selection of jams to pick from, one with six jams and the other with 24 flavors.

While people liked the table with more varieties of jam, they didn’t buy nearly as much jam as those did with fewer options.

That’s because the more choices we get, the harder it becomes to pick something out. In the end, most people will just walk out empty-handed.

The same thing applies to your links. The more you have, the tougher it becomes for the email reader to decide which ones to click on. In the end, they may not click on any of them.

The second reason we recommend you cut down on links is aesthetic but still viable. It’s simply because too many links look spammy. Your email could get stuck in the spam filter instead of the recipient’s inbox.

2. Email Send Day/Time Testing

As we’ve written about a few times on this blog, Tuesday, Wednesday, and Thursday are your best days to send emails. The most ideal times to send emails are early in the morning, midafternoon, and/or later in the evening.

best time to send emails

Image courtesy of CoSchedule

Okay, so if that’s all so concrete, then why bother with email marketing testing? Well, because those times and days can fluctuate slightly depending on your audience.

Even once you think you know the best days and times your audience is most receptive to your emails, it could change. This is something that will always be in flux, and you have to be prepared accordingly.

At least monthly, you should A/B test when you send your emails. Then, accommodate the latest preferences of your audience.

3. Email CTA Testing

Ah, the call to action. It’s so necessary, but is it working for your company?

Your email CTA can appear in several formats. Perhaps it’s a line of text at the end of your emails. Maybe you’re a bigger fan of the CTA button, so you park it front and center in your email.

There’s proven research that shows the CTA button is more effective than a link in emails. According to data from Campaign Monitor, it’s possible to see a 28-percent boost in your click-through rate with a button.

That might be the first thing you test, whether a text CTA or a button performs better for your audience.

Once you make that decision, you’re not done yet. You should also test for the following:

  • The amount of whitespace your email has, which can either obscure your CTA button or let it stand out
  • The color of your CTA button as well as the other colors in your email, including the background; these should be two separate tests
  • The design of the button, like whether you include effects like shadows or not
  • The size of the button
  • The placement of the button
  • The CTA button copy

4. Anchor Text Testing

Are you using anchor text in your emails? If not, you’re missing out on a great way to pique the interest of your audience.

What exactly is anchor text, anyway?

This is a hyperlinked copy that often ends in an ellipsis. It’s a preview of what’s to come that’s meant to tantalize the audience and draw out their natural sense of curiosity.

Here’s an image that showcases how anchor text can play to several emotions.


Image courtesy of Yesware

The first option you have to is to get your audience curious. If this is the approach you take, then the accomplishment you highlight should be something they themselves would want to do.

Your company also has to be well-known enough that if mentioned by name alone, people know who you are. Make sure you don’t give the whole secret away in your anchor text.

You can also invoke empathy in your audience, but you’d have to know their pain points to do this.

Maybe your audience is having a tough time converting leads or getting anyone to read their blog. By showing you’ve had the same problems and were able to overcome them, you display your empathy.

Finally, you can get your audience scared by making them feel like they’re missing out on something awesome. That’s what FOMO is all about, after all.

No matter which of the emotions you go for in your anchor text (and maybe it’s another emotion entirely), remember this is just a preview. Once the reader clicks the link, they’ll get the whole story.

5. Personalization Testing

You know you should personalize your emails, but how much personalization is enough? Only A/B testing can answer that question.

Here’s a great example of a travel company. You can see two personalized emails they tested side by side.

A/B testing

Image courtesy of Vision6

In the first version of the email, it isn’t super personalized. Yes, it says “featured hotels just for you,” but that’s about as far as it goes.

On the other side of the email are all the exotic locations a customer might want to travel to. It’s a good email, but is it great?

Not as great as the second email. This one has the name of the email recipient outlined in the sand. Their name is also in the first line of the body beneath the image.

How excited would you be to see your name traced in the sand? Very, right? That alone is going to make you want to read through the email.

Get creative with your personalization. Whether you do something like this travel company or you make your audience feel unique in some other way, A/B test it.

6. Tone Testing

How do you write your emails?

Perhaps you take on a more informal tone, such as the voice of this blog post. It’s relaxed yet informative and feels like talking to a friend rather than a colleague. You’re reaching out to your email audience on a more personal level.

Maybe you prefer the formal approach. The writing is more buttoned-up and may include jargon and terms that only those in your niche would understand. Both are appropriate in some instances and industries.

Also, your tone might vary depending on which audience segment you’re trying to reach. You might write in a more relaxed tone with your established customers and more formally with your new ones.

The opposite could also be true.

The best way to know what your audience will like best is through split testing.

Prepare two versions of the same email, one with an informal tone and the other written more formally. Then see which one gets the highest open rate.

Testing for email tone is not a one-and-done thing. You’ll have to keep repeating the test when sending out major emails to new and established customers alike.

7. Content-Length Testing

No one likes slogging through a novel of an email. Going in the opposite direction can be risky too since brevity can also be vague if you’re not careful.

We recommend testing at least three versions of an email. One should be the long one you prepared with love and time.

The other should be a mid-length version that’s shorter but not too short. Finally, try to condense your message into the lowest number of words possible.

Now test all three emails to see which gets the highest open rates.

We’d bet it’s probably the mid-length one. That’s not to say that longer emails never have their place. In a newsletter email, for example, your audience expects a longer, meatier message.

Short emails are also appropriate in some instances. If you’re sending a reminder about an upcoming event or webinar, this wouldn’t be very long.

Are you having a hard time cutting your emails down to size? You could always try adding some images to break up longer blocks of text.

2018 data from HubSpot found that many consumers (65 percent) liked seeing images in emails more than they did text.

Have you written about the subject before in a blog post? Then link to that rather than devoting paragraphs to the same topic in your email.

This lets the audience choose if they want to do further reading on the subject or not. Just remember not to add too many links.

8. Link Text Length Testing

Speaking of links, there are so many ways you can format them.

For instance, let’s say we’re linking to our EngageBay homepage. We could just say “check out EngageBay for all your marketing needs” and link to the name of the company. Alternately, we could do something like this: “if you need an all-in-one sales and marketing software…”

When you add hyperlinks to your email, how should you do it? How many words should be included in the links? This is something else you’ll have to test for yourself.

Most marketers agree that 12 words are the max amount. Here’s a MailChimp illustration that shows that data in more depth.

words per URL

Image courtesy of MailChimp

It appears the more words you cram into your email links, the lower and lower the email open rate is.

9. Greeting Testing

Did you know the very opening words of your email can be a big determinant of your campaign’s open rate success? It’s true.

Whether you say “hello,” “howdy,” or “good morning” does matter. How important this all is to your audience is up to you to find out through A/B testing.

Like the body of the email, the tone of your email greeting will vary.

If this is a new customer and you’re keeping a somewhat formal tone, you might write “welcome” as your greeter. For older, established customers, starting your message with a “hey there” is perfectly appropriate.

What seems to matter most in terms of open rates is greeting personalization. Instead of “hello,” try “hello, [name].” Here are some good examples to get you started:

hearts and tails

Image courtesy of Constant Contact

My Giftlist

Image courtesy of Oracle


Image courtesy of Buffer

10. Subject Line Copy Testing

One of the most important parts of any email campaign is sending messages with stellar subject lines. This is the first impression you make on your email recipient, after all. It has to be a good one.

There are lots of ways to craft a fantastic subject line.

Personalization is always good. Stats and figures can pique interest. Discounts and deals get most people clicking. Your company might even support the liberal use of emojis in subject lines.

Emojis by subject line

Image courtesy of MailChimp

Whatever works for you, you have to know for certain. An A/B test will truly come in handy here.

Your subject line, like your anchor text, can connect with your audience on an emotional level.

Once again, you might try to come across as empathetic. You can also foster that sense of FOMO by sending an email about a deal that’s only good for today or a sale that’ll last 12 hours more.

You can try to get your audience curious by writing a subject line much like you would your anchor text.

You make a partial statement, ending it with an ellipsis. By telling only part of the story, you hopefully intrigue your reader enough that they want to click, open the email, and see how the ending goes.

11. Subject Line Length Testing

Okay, so you’re confident the content of your subject line will appeal to your audience. You A/B tested it and everything. You’re all ready to send the email, right? Not so fast.

You should also test for the length of your subject line before you launch your campaign.

Exceeding 50 characters is no good, as there’s a chance the audience member might see a cut-off version of your subject line. This depends on the device they’re using, of course.

On smartphones, your audience is more likely to see the crudely chopped subject line. If they have a Galaxy S4 device and use it in portrait mode, Yesware says the email will be cut at just 33 characters.

If they’re an iPhone owner who uses their phone in portrait mode, they get 35 characters, maybe 38. By flipping their phone to landscape mode, it’s 80 characters, but who does that?

Five words are the sweet spot. This should fit the character count requirement so that even if your audience is on their phone and holding it in portrait mode, they should see the whole subject line.

Automate The A/B Testing Email Process with EngageBay


A/B testing in email marketing is a crucial part of your campaign’s success. It will influence email open rates and click-through rates as well.

By starting an A/B testing campaign, you can be sure all elements of that campaign resonate with your audience.

Nothing is too small to split test. From the placement of your CTA button to which time to send an email, it’s all worth it. Some elements of your campaign will have to be tested regularly and others just once or twice.

Knowing your audience allows you to create stellar, winning campaigns. A/B testing ensures all parts of your campaign are tailored to their interests and pain points, winning you loyal, buying customers.

Leave a Comment

Your email address will not be published. Required fields are marked *

Scroll to Top