Test Your Mettle, Not Your Ads

Forget the Studies -- Consumers Will Yield True Results

Posted by Bart Cleveland on 05.11.07 @ 03:31 PM

Your client wants to test creative. What's your response? It's a tough question: If you refuse, you risk sounding afraid of the results. If you agree to test an ad, whether you realize it or not, you've compromised your client's best interest.

My partner, Steve McKee, heads up our clients' brand strategy development. Clients often ask him about the value of testing creative. While we do recommend clients research several components of strategic development, we advise against testing creative concepts. If a client is truly committed to differentiating their brand, Steve says, creative testing will serve neither as an assurance nor a determinate of success. In fact, testing can be detrimental to a brand. Steve's conclusions, mentioned below in brief, are based on his own experience, as well as similar positions of many experts in our field, including marketing research firms and leaders from such brand icons as Nike and VW.

When asked by a client about testing creative, we advise against it for a very simple reason: The science just isn't there yet. A few years back, Ad Age ran a story about the vagaries of ad testing. They cited Volkswagen's popular 1997 "Da, da, da" Golf commercial, which some in VW management didn't want to air. The article explained how GM had tested a pair of commercials for Volkswagen of America using a custom-designed pre-testing system from Design Research International, one of the nation's leading market research firms. According to the report, the successful VW commercials flunked under GM's process.

GM might be able to afford the most sophisticated ad-testing system in the world, but even stats can't predict real-world success. Replicating the reaction of a few guys swilling beers during a Saturday afternoon ball game into a research setting remains a near-impossible feat.

When you invite someone into a research study, whether it's an online survey, a focus group or even an in-home observation, his or her behavior will change simply as a function of "being researched." They know they're being watched; they may even believe their job is one of critical analysis.

Think about how a focus group works. People are invited and paid an incentive to offer insights and opinions useful to the sponsoring marketer. The pressure to contribute something of value is immense. For a participant to admit he or she enjoys an ad at face value, or to admit it might influence a future purchase, is rare. Instead, participants tend to overstate how underwhelmed or emotionally charged the ads are, as if this serves to assist the screeners.

A desire to contribute isn't the only problem. Even an honest opinion may be difficult to communicate. According to experts on the leading edge of this stuff, people just aren't able to articulate or possess full knowledge of all of the ways advertising affects them.

There are times an agency isn't given a choice to test creative. This doesn't mean we should expect testing to yield bankable results. Anyone worth their salt in this business knows there is only one way to accurately test an ad's effectiveness: Run it.