Into the Pool

By Published on .

Cyberpolling has come of age - at least in the eyes of Wall Street. Harris Interactive went public last December, the first company to sell stock mainly on the strength of its work in polling on the Internet.

So does this IPO mean that online polls are ready to dethrone telephone surveys as the "right way" to conduct public opinion research? As the rental car ads say, "not exactly."

Online polling has actually come a long way in the past 18 months, with a variety of companies trying to use the Web as a data collection tool for a wide range of studies. And some creative folks have thought up ways to use online channels that will work just fine for some types of surveys. A number of polling firms are conducting online surveys of well-defined groups, for example.

Whenever you can define the population you want to survey in a limited and precise way - human resources administrators, congressional staffers, mayors of U.S. cities - you can consider collecting the data online. Of course, the members of the group must have Internet access and be familiar enough with e-mail or the Web to use the medium to take the survey. At this point, the best methodology is to obtain or create a list of the entire population, draw a sample from the list, and ask those chosen to participate. In a sense, this type of online poll is just a new version of the mail survey - faster, but the same basic approach.

But this method does not work for the general population yet, and the reason is simple. About half of the population still has no regular access to the Internet, and they are demonstrably different from those who do: They are poorer, less educated, and more likely to be members of a minority group than those with access.

InterSurvey in Menlo Park, California, has come up with a creative way to survey the general public using the Internet, and the idea is astonishingly simple: Draw your sample of households in a rigorous, scientific way, and then give every household free Internet access through WebTV; those who receive the service agree to be polled by InterSurvey via e-mail. This allows for both a good general population sample and control over the environment for administering the survey online. InterSurvey can even show a television commercial or other video clip to the respondents and then question them about it. The approach, still in its infancy, faces some substantial hurdles, but it is very promising, particularly for market research applications.

And that brings us back to Harris Interactive, which has recruited a massive panel of more than 4 million people online who say they are willing to participate in online polls. Harris draws a sample from the database, asks them to fill out a questionnaire online, and then tabulates the results. In the past two years, Harris has conducted a large number of online polls simultaneously with telephone surveys using the same questions. Over time, they have developed formulas for weighting the online surveys to match the general population surveys. And thus, they say, the online surveys are just as good as telephone polls - better, even, in that they are faster and cheaper.

To make their point, Harris spotlights a series of pre-election polls the company conducted online in 1998, noting that they correctly identified 21 out of 22 winners in the races covered. It's a pretty good record for pre-election surveys, although their claim has been challenged by many in the field.

The basic problem with Harris' work - in addition to the problem of drawing a viable sample online - is that half of the general public simply has no regular access to cyberspace and therefore can't be polled online. And since the people online are different from those without access, this problem is essentially insurmountable. In addition to demographic differences, there are other disparities that could be significant.

Forget those problems, one might say: Those online election polls from 1998 looked pretty good, right? Harris Interactive thinks so, and they have announced their all-Internet Election 2000 project, "nothing less than the largest privately funded polling program in history." But how did Harris do in two 1999 off-year gubernatorial elections in Kentucky and Mississippi?

Not so well. Harris got the winner in Kentucky right, although the margin was overstated. In Mississippi, the online poll said Republican Mike Parker would beat Democrat Ronnie Musgrove, 53 percent to 39 percent. The final results? Musgrove 49.62 percent; Parker 48.52 percent. Oops.

No poll is infallible, of course. But everyone pushing a new technology like online polling has a long way to go to demonstrate its validity and reliability.

Most Popular