E-mail testing

Published on .

Marketers that would never send out a direct mail campaign without doing some form of testing often don't think twice about sending out an untested e-mail campaign.

Only about 40% of all e-mail marketers use e-mail testing strategies on a regular basis, said David Daniels, lead analyst with Jupiter Research. B-to-b marketers test even more infrequently, said executives at several e-mail service providers. Jordan Ayan, founder and CEO of SubscriberMail, an e-mail marketing service provider, said only 20% to 30% of his b-to-b clients actively test.

Yet marketers that do test their campaigns see an incredible return, Daniels said. "Marketers that test mailings are more likely to have e-mail conversion rates that exceed the industry average of 1% to 2% compared with marketers that do not test," he wrote in a recent report. "Forty-seven percent of testers report average unique conversion rates of more than 3%. Only 34% of nontesters attained the same level of results."

If the benefits of testing are obvious, why don't all marketers do it? Usually, it's a lack of resources and knowledge. Marketers know they should be testing but think they don't have the time. Even more important, they don't know how to get started. The process, said Katie Cole, VP-analytics & research at e-mail service provider Quris, starts with defining several actionable objectives.

"First define your marketing objectives, [for example,] increase conversions for product `X'. Then determine the tactics to drive that behavior: for example, two different URL placements, two offers varying by richness, timing of the send-Friday versus Saturday-[and] targeted segments," she said.

rule out deliverability issues

Marketers should also rule out all deliverability issues, running messages through spam filters and creating seed addresses to make sure messages are being delivered and formatted correctly, and in a timely manner. This also includes making sure your message appears the same for every e-mail client it might touch and that all the links in your copy work.

And then the testing starts. According to a recent Jupiter Research report, 66% of the top 30 e-mail service providers offer testing capabilities; this means the majority of marketers can do their own testing using their ESP's software. But even those who don't have testing and reporting capabilities can get involved, and the good news is you don't have to do complex multivariable testing to see big benefits. You can start seeing results quickly with simple A/B testing, sending out e-mail to two groups with a single variable altered in each.

"It's easy to split your [test] list in half and, within 24 to 48 hours, have 95% of the responses that you're going to get back," said Loren McDonald, VP-marketing for e-mail service provider EmailLabs.

McDonald suggested using 5% of your list to create a test sample. That said, if you have a smaller list, as many b-to-b companies do, you may have to use more of your list for the test sample. The general consensus is that anything under 100 responses isn't going to be statistically significant.

But be warned, Ayan said, results will only be significant if you do a true A/B split."Marketers think they are doing an A/B split but then send out group A in the morning and B in the afternoon," he said. "You can't look at results when you've got two groups going out at different times."

What to Test

Most marketers have dabbled with changing their subject or "from" lines, which are two of the most important tests when it comes to boosting open rates. (Day and time are also significant.)

Steve Morse, VP-marketing at ClickTactics, a provider of marketing automation solutions, suggested getting creative with your testing. One ClickTactics customer recently tested personalization that included placing a photograph of a sales rep in the body of an e-mail. "The messages with the pictures have stronger conversion rates," he said.

Geene Rees, director of marketing strategy at Digital Impact, an online marketing service provider, suggested testing other personalization features, such as naming the prospect in the message. He also suggested testing text and image placement within the message. "One of our clients moved their images to the right and the text to the top and saw a jump in conversions," he said.

And marketers should never forget that conversion rates are going to depend on what you're selling, said Shannon Coulter, senior marketing manager at Lyris Technologies, a provider of e-mail marketing software. "One thing that I don't see nearly enough of is testing the offer. The rule of thumb for traditional direct marketing is that success is 40% list, 40% offer and 20% creative," she said. "It's not much different for e-mail marketing. Testing the day of the week and HTML-versus-text is great, but a significant factor in any campaign's success rate is whether or not the offer itself resonates."

You can get to the bottom of that by polling your readers directly and asking them what they liked and didn't like about your message; also, what they need but didn't see.

When you've tested all the individual elements of your campaign, you can start getting into multivariable testing, which looks at how multiple elements interplay and affect each other.

Once you get into the habit of testing, you'll want to stick with it, McDonald said. He suggested testing every message that goes out. "You should always be testing if you have the time, and build on prior knowledge," he said. "Maybe you start with testing the day of the week. Then you come to a conclusion that this is the best day so then you test time of day. And then you test subject lines and format and type of content." 

Most Popular
In this article: