“Test and Learn” is a ubiquitous phrase in any strategic marketing plan. We all know that brands should be testing all components of their marketing efforts, from brand positioning to creative messaging, to offers to the target audience, to marketing channels, to data sources. Yet, despite instant access to an enormous amount of data, many marketers have lost sight of what it actually means to truly “test and learn.” Too many brands merely pay lip service to the idea.
I spent a good part of my career in the direct mail world where testing and learning was everything. One of my biggest clients was Reader’s Digest, a company that was revered for its prowess in this area. It set standards that everyone wanted to emulate. There was a war room on the company’s sprawling campus where every piece of mail ever sent to consumers hung on the wall with various phrases highlighted to indicate what had been tested and with detailed notes indicating how each had performed relative to the control. Similar rigor was also part of the marketing operations excellence at other leading direct marketing clients including Columbia House, Time Life and Book-of-the-Month Club.
In a single piece of direct mail are a myriad of elements that can be tested against a standard control letter. Sure, changing the cost per issue or the number of “freebies” that come with a customer’s purchase could drastically change conversion—but so could subtle changes in copy, especially the copy customers could see before they even opened the package. Letters often had a highlighted box (called a “Johnson Box” after Frank Johnson, who invented this along with other classic direct mail techniques). A company might test 50 or 60 different phrases in this box to see what grabbed consumers’ attention. Believe it or not, a change in the use of words could yield a 10% to 15% lift in response rate.