The danger in ad recall tests

By Published on .

In the film "Manhattan," Woody Allen asks his ex-wife, played by the young and captivating Meryl Streep, how she could have left him for a woman. She snaps back that he knew her history before they married; even his analyst had warned him against her. With his typical high-strung earnestness, Woody retorts, "Yeah, that's right, but you were beautiful, so I got another analyst."

As an analogy for what prompted us to spend a year doing research on research, you can't beat that single snippet of film for being, to quote our British friends, spot on. Watching the ads we liked best condemned to the dark based on bad recall scores, we decided to finally challenge the "analyst." We hoped our faith, and our investment, would not prove, like Woody's, to be in the end misplaced.

We tested 60 of Unilever's ads under three major ad-testing systems-Ameritest, Ispos-ASI and Millward Brown-to get insights into what they were actually measuring.

A VP of consumer insights and a risk-taking researcher found much in common working together. There was our mutual mistrust of recall, neither of us believing a verbal-based measure could ever successfully calibrate the role of emotion in ads-that powerful driver that not only incites purchase but, as it turns out, builds brands. We were convinced that liking and emotion are deeply linked, based on past research and our own common sense. And we both need facts, not just our hunches, no matter how good it turns out they are.

Of all we learned examining the differences among the three systems, the significant negative correlation of -.39 in our sample between day-after recall and liking left us with one "headline" conclusion: Recall completely misses how emotion is working in ads.

And if advertisers are not rewarding commercials that carry the emotional content that ultimately is what really builds brands (that other key finding of our experiment), then they are throwing away long-term branding along with their agency's most creative ideas.

episodic memory is key

This, minus the charts and the graphs, is the paper we delivered at the Advertising Research Foundation conference in New York this spring. As we strive as advertisers to not only break through the clutter but to create loyalty, ownership and, frankly, a bald-faced love of our brands, the need has never been greater to understand that complex thing called "brand image" that resides, not in the rational verbal-based memory, but in that episodic memory system, the one that makes stories starring you and the brand. That's the one that's driven by emotion, the one that's connecting people to product.

Recall, a measure based on a respondent's ability to recognize an ad from a verbal description, favors rational, informational, benefit-driven ads, which frequently follow the "early and often" rule of brand mentions.

As an advertiser, if you're using recall to make the cut among advertising executions, it is those ads you're choosing-those that our respondents called boring and ordinary. You are using a verbal-based system, a test of copy, to make decisions in a visual medium, a medium that delivers those powerful images that set up house in the human mind.

Clearly, there's more than one way to measure breakthrough. Recall is undoubtedly an appropriate measure for certain ad applications. However, by rewarding advertising with a strong rational and informational focus, we suggest the usefulness of recall scores is severely limited. The paper "Why Day After Recall Misses the Emotion in Advertising that Builds Brands," which offers the detailed background of our research, is available for download at

As marketers, agency professionals and researchers working in one of the most creative fields out there, it is incumbent on us to examine existing paradigms with some of the same muscle we bring to bear to evolve the advertising itself. Challenge. Question. That's what keeps our industry exciting and sometimes even revolutionary. It also happens to make us better at what we do.

What's not measured

The results of this experiment reaffirmed to us that we can use ad research to do what it has always claimed to do-engender the best creativity. But it's critical to understand the science we're using, to acknowledge what its practitioners are measuring and what they are not, as we seek to understand that complex thing called memory, which knows a product's stories and holds onto brand messages.

Decoding the role of emotion in that process goes on. But the connection between emotion and brands cannot be denied except at our peril. While Woody's emotional faith may have been misplaced, no one can argue its strength-a fact we will do well as advertisers to file in our own long-term memories.

John J. Kastenholz is VP-consumer & market insights, Unilever Home and Personal Care North America, Greenwich, Conn., and Chuck E. Young is president-CEO, Ameritest/CY Research, Albuquerque, N.M.

Most Popular
In this article: