He's part of a growing nightmare for marketers-consumers with opinion fatigue who reject almost all attempts at polling. It's a problem so big it brought 30 of the top executives in market research to Chicago on Sept. 28 for a roundtable at the Research Industry Summit for Improving Respondent Cooperation. The gathering was hosted by the Institute for International Research in cooperation with RFL Communications, publisher of Research Business Report, and CMOR.
There, the heads of the five leading global research companies and top research executives from the likes of Procter & Gamble Co., General Motors Corp., IBM and McDonald's for four hours hashed over a problem stunning in its scope, if uncertain in its impact.
After all, no one really knows whether people who don't answer surveys are similar to those who do, because they don't answer surveys. But the industry does know nonrespondents tend to be disproportionately male, black, Hispanic and young (30% of households headed by consumers 25 and younger now only have cellphones and are impossible or highly expensive to reach by phone).
Response rates under 10%
Though no truly global figures are available, almost every researcher has seen participation erode in recent years, with rates under 10% increasingly common. Surveys tend to poll the same people over and over, often "professional respondents" who go hunting for research for dollars.
Just 0.25% of the population supplies 32% of responses to online surveys, said Simon Chadwick, former head of NOP Research in the U.K. and now principal of Cambiar, a Phoenix consultancy, citing research by ComScore Networks. More broadly, he said, 50% of all survey responses come from less than 5% of the population.
That leaves lingering suspicions that survey research may be getting less reliable. "We're perpetuating a fraud," Mr. Chadwick said.
Yet when ComScore presented its findings that only a thin slice of the population accounted for the majority of research, it couldn't sell marketers on paying more for a panel free of professional respondents, said Gian Fulgoni, chairman of ComScore.
"It's like the hole in the ozone layer," said Shari Morwood, VP-worldwide market research at IBM. "Everyone knows it's a growing problem. But they just ignore it and go on to the next project."
Not all marketers think it's a problem. "A low response doesn't necessarily lead to a biased response," said Michelle Salazar, VP-global brand and business research at McDonald's.
'We don't know'
"You may be right, but we don't know," said Jim Lochrie, general director-North American marketing research at GM. The automaker can consistently get the same results from two surveys, he said, but that doesn't mean both aren't biased the same way.
P&G can't always get even that much reliability, despite spreading $200 million among 600 vendors who do its survey-related research. Kim Dedeker, VP-consumer and market knowledge at P&G, presented one example in which online and mail surveys on an instant-coffee concept came up with diametrical results.
"If I had only had the online result in this particular case, I would have taken a bad decision right to the top management," she said.
In another case, two surveys a week apart by the same online researcher yielded different recommendations. "We're having tremendous issues moving from concept to launch," Ms. Dedeker said. Research that qualifies projects for millions of dollars in advertising and capital investment sometimes is contradicted by other studies just before rollout.
While she was careful not to blame online research or specific vendors, she said the problems boil down to "the integrity and methodology," with respondent-participation problems one possible factor. "I'm not sure we're aligned on the nature of the disease we're treating," she said.
Nor were participants aligned on a solution. Online research -- once touted as a way to improve respondent cooperation -- now may be making it worse. While it's easier to respond to online surveys, it's also easier to crank them out, leading more consumers to tune them out, said Patrick Glaser, director-respondent cooperation for the Council for Marketing and Opinion Research.
Bill Lipner, chairman-CEO of Insight Express, suggested a $50 million industry war chest to market the importance of participating in market research, which several participants said would be impossible to raise and possibly ineffective.
VNU's Nielsen Media Research has actually seen respondent rates rise from 36% to 45% the past five years, said Paul Donato, chief research officer. That's largely because it pays respondents handsomely for their two-year commitments -- so handsomely that Mr. Donato acknowledged that some on the Media Research Council think it may bias results -- allowing panelists to buy cable subscriptions and DVRs.
Ironically, no one in a roomful of market researchers suggested researching what might best persuade nonrespondents to participate, though Dennis Murphy, VP of the technology practice at Directions Research, said it's time to find out how different nonresponders really are from responders -- something largely neglected since the 1970s.
Several participants suggested making surveys shorter and less cumbersome. Mr. Pairan said he'll sometimes answer surveys -- up to three questions. After that, he tells the researcher his doorbell is ringing and hangs up.