You've Got Surveys

By Published on .

Most Popular

Innovative approaches to market research highlight the industry's move onto the Web.

When Richard Hatch, the million-dollar winner of CBS's Survivor contest, joined his fellow castaways for a live reunion show, following the airing of the final episode this past August, Bryant Gumbel had some bad news for him: The viewing public did not approve of his victory. How did Gumbel know this, just minutes after the end of the series' broadcast? From a Web survey taken immediately following the show. "This is the first time I've seen survey results actually applauded," says Kathleen Frankovic, director of surveys at CBS News, referring to the audience's reaction at the post-show reunion of the original contestants.

CBS's Survivor survey illustrates the benefit of speed that the Web can offer companies conducting market research online. An increasing number of marketers are recognizing this, and other benefits, and are moving more of their research onto the Web. According to figures compiled by Inside Research, an industry newsletter, the value of online research conducted by 29 of the largest market research companies, has grown from just $3.5 million in 1996, to $254.8 million this year (these firms represent 90 percent of industry revenue in this arena). While this is still a small percentage of the 1999 total U.S. market research industry revenue of $4.8 billion, (as reported by the American Marketing Association's Marketing News), it represents triple-digit percentage growth in each of the past four years for online research.

The advantages of using the Web include lower costs, the ability to survey hard-to-reach respondents, and of course, speed. Traditional forms of research have been plagued by decreasing levels of cooperation from the public, and the refusal rate of people asked to participate in telephone or in-person surveys has increased from 49 percent in 1978, to 60 percent in 1999. With Web surveys, people can be invited to participate via e-mail, and then decide when to complete them, at their convenience. What's more, the Web opens up the use of rich media, such as streaming video, to test ads and movie trailers, something not possible over the phone.

One company that has become a true believer is General Mills, which last year conducted 20 percent of its research online. That number is expected to grow to 60 percent next year, as the company believes that by using the Web, it is saving as much as 50 percent in costs, and speeding up the research process by up to 75 percent. "Surveys that used to take four weeks can be completed in eight to 10 days," says Gayle Fuguitt, vice president of consumer insight at General Mills.

However, as with many new techniques that challenge tradition, not all marketers or researchers are convinced of the Internet's utility in this field. Among the criticisms: Online research is restricted to those who are already online, and that it doesn't represent a broad cross section of Americans. "There's a lot of hype out there on the supplier side, but still a lot of caution on the buyer side," says Barry Goldblatt, executive director of marketing research at Johnson & Johnson. "There are a lot of questions still out there about this methodology."

Market research firms, both old and new, are utilizing unique approaches in addressing some of the criticisms, and making research on the Web more relevant than ever to marketers. Here's a look at what some are doing.

The Nielsen of Web Surveys Knowledge Networks, which conducted the Survivor poll for CBS, has created a panel of over 100,000 consumers from scratch. These consumers are each supplied with free Web TV devices and monthly Internet access. The panel, which is expected to grow to 250,000 by 2001, is chosen through random digit dialing technique, which gives every household in the U.S. an equal chance of being selected. This ensures that the panel is a scientifically valid cross section of the U.S. population, allowing Knowledge Networks to provide research information on any particular portion of the consumer market, or the market in its entirety, via their online surveys.

The creation of this reliable probability sample by Knowledge Networks was one reason CBS News also hired them to conduct polls on immediate public reaction to President Clinton's State of the Union address last January and the Gore/Bush debates in October. "As a news organization, we have specific requirements when it comes to accuracy and speed; we have to put our name and reputation on those polling results," says Frankovic.

What's more, the market research potential of this panel goes way beyond surveys. Utilizing the Web TV device, Knowledge Networks plans to start monitoring the television viewership and Web usage of its panel, as well as tracking print media consumption through surveys. Software loaded onto panelists' home computers will be able to track Web surfing there as well. In addition, Knowledge Networks announced in September, the acquisition of Promotions Decisions Inc., a company which monitors consumer purchase patterns in supermarkets. These moves will allow the company to provide a "360 degree" view of consumer attitudes, brand preferences, opinions, demographics, interests, hobbies, television viewing, Web surfing, purchase intentions, and actual purchases, all from a single source of panelists.

"Eventually we will have thousands of data points on each of our panelists," says Catherine Edwards, vice president of marketing at Knowledge Networks. This mountain of detailed data will be a gold mine of insight into consumer behavior and thought, offering marketers an almost "Big Brother" peek into the minds of consumers. A company that was seeking to launch a new skin care toner, for example, could test new concepts through an online survey, and analyze the types of people most likely to try a new product. Different advertisements and packaging could be tested on the most receptive group, and data on the media outlets and Web sites favored by the group could be compiled. Once the product was launched, the effectiveness of the marketing and the actual number of purchases being made of the product-and by whom-could also be tracked. All from one statistically reliable data source.

Nice 'n Easy InsightExpress takes a different approach. Rather than attempting to create a system to integrate the offline population into its sample, the 1-year-old company is using the speedy and economical data collection capabilities of the Web to provide inexpensive, automated, do-it-yourself research. With prices starting at $450, and surveys completed in 6 to 7 hours, InsightExpress is leveraging the power of the Web to bring market research abilities to companies that previously may not have considered it as a viable business tool. "We want to make market research as pervasive as Microsoft Excel," says Charles Hamlin, president and COO. "Let's bring some consumer input into every important business idea and decision."

While Hamlin agrees that you can't use Web surveys to conduct research on the population at large, he points out that there are plenty of marketers and research projects for which an Internet sample is adequate, or even desirable. "If you are Volvo, do you really care about people making under $30,000 who are not online?" he asks. "The online world can provide a representative sample of many populations, for many clients who are seeking a certain demographic." This is especially true for dot-com companies, who are usually only interested in the opinions of the online population.

Clients design their own surveys with help from templates on the InsightExpress Web site. Respondents are recruited by Hamlin and his team, through banner ads on targeted Web sites, and then screened to obtain the desired demographics. The automated nature of the service keeps costs down, and clients can often start receiving responses to a survey within minutes of downloading to Insight's servers. "The speed of this type of research is unbelievable," says Lisa Manuzza, director of research at, a portal site. "And the cost of these surveys is such that I can test 10 times as many ideas as I could with traditional research." In fact, in her most recent study, Manuzza used 34 cells, whereas with a more expensive survey she might have only used three or four. As a result, recently moved forward with a concept that they probably would never have tested before the advent of InsightExpress.

Better Surveys through Science? Not every online research firm believes that the sample used in online research precludes surveys that can reflect the population at large. Harris Interactive differs from Knowledge Networks and Insight- Express in that company executives believe they can accurately weight the results of online polls to incorporate the offline world into their findings, using a statistical technique called "propensity score adjustment."

"We can take all the speed, cost, and interactive advantages of Internet research and use weighting to provide high-quality data on both the online and offline populations," says George Terhanian, vice president for research and methodology at Harris Interactive.

Each month, Harris runs parallel telephone and Internet surveys, where they compare a representative sample of the U.S. population with the panel of 6.6 million people the company has recruited to take part in their online surveys. Using analysis of these two identical surveys, Harris weights its online surveys for that month to reflect the general U.S. population, using demographic figures obtained from the Census Bureau's `Current Population Survey."

While some in the research community are not convinced that Harris Interactive's methodology is valid, Paul Rosenbaum, an expert professor in the area of propensity score adjustment at the Wharton School of the University of Pennsylvania, believes it is theoretically sound. However, Rosenbaum warns: "How successful you are depends on whether or not you can observe all the variables involved." With the many factors involved in understanding who comprises the offline and online populations, this could be a tricky proposition.

Whether or not Harris Interactive is completely successful in applying propensity score adjustment in its work may be debatable. But the company's methodology passes muster with a growing number of clients. Last year Harris was the fastest growing major research firm in the U.S., with revenue up 34.2 percent over 1998.

The contrasting approaches taken by different research firms have their supporters and detractors, but almost everyone in the online research business agrees that the future of the industry is on the Internet. Larry Gold, editor of Inside Research, estimates that in the next three years as much as 60 percent to 80 percent of research will be conducted via the Web. This should allow enough leeway for each company's approach to serve marketers and their various needs and budgets.

Now if they could only do something about moving all those telemarketing calls onto the Web too.

In this article: