Online testing rated

By Published on .

Most Popular
With half the adult population having online access, and with the unlimited capacity of the Internet to influence our lifestyles, it hasn't taken long for package-goods companies to realize the Web's potential as a tool for understanding and anticipating consumer needs.

The Web, after all, is governed as much by brands as by bytes and bandwidth, making it a natural place to gauge consumer thinking and behavior.

As the Internet casts its glow over commerce, online testing is becoming an increasingly popular alternative and adjunct to traditional market research methods, such as mall-intercept interviewing. And package-goods companies are taking advantage of what the Web has to offer -- from speedier results to global reach -- so they can race to market and gain competitive advantage.

But the Internet isn't all advantage. It has serious drawbacks, too, some of which pose substantial risk to companies that naturally want to keep innovative ideas private at least until launch time.

When it comes to brand-new-product concepts, the Internet has the potential for damaging leaks. Private product images can be posted on the Web -- to circle the globe in milliseconds -- by anyone who has access to them. Such leaks can damage or even destroy a product's chances even before it is launched.


So what are the pros and cons of online consumer testing? And will the Internet transform market research as we know it?

nThe most obvious advantage is time savings. Online concept testing can shrink testing time from four weeks to seven days or even less -- a definite plus in an industry where some products are tested for six years prior to national launch.

At Unilever -- which markets dozens of everyday beauty, homecare and food brands, including Dove soap, Vaseline skin lotion and Lipton tea -- testing time has been improved dramatically due to the Internet, reduced in our experience by at least 25%. On a recent laundry concept screen, we got results in six days from receipt of concepts.

nNot only is online testing faster than conventional testing, it is generally less expensive, especially when sample sizes are large. On the qualitative front, a typical online focus group can run from $4,000 to $5,500, with the high end including written reports and analyses. Its real-world counterpart often results in a hefty travel budget for the marketing staff, which could add thousands of dollars to an initial $3,000 to $5,000.

Internet technology itself offers marketers several other distinct advantages.

nOnline testing can be limited to a few Internet users or reach a global audience. Thus, intercept testing of teen-agers in, say, a California mall can be expanded almost infinitely to include Web-surfing teens throughout the U.S. or even abroad.

nEqually important, the Web enables marketers to reach their target audiences quickly or to survey consumers from wider cross sections of the public just as easily and efficiently. It even allows companies to provide customized advice on product usage and satisfaction, even for low-incidence items, a benefit that formerly was time consuming or more trouble than it was worth.


nThe Web's potential for cross-country dialogue among would-be consumers can give companies clear ideas on buyers' likes and dislikes, helping to shave pre-launch preparation time while providing valuable feedback that formerly took months to collect.

Quick results are what all marketers want -- and this is a prime benefit of online testing. But longer test periods are possible online, too. Internet tests can be extended for as long as marketers want; questionnaires can be posted on the Web for indefinite periods. Results can be reported daily -- as easily as with the click of a mouse. They can even be e-mailed automatically to marketers. A danger here, though, could be the temptation to run with results that are incomplete or not fully analyzed.


nAnother benefit of online testing is that participants become engaged, more so perhaps than in other test methods. Usually, they are highly motivated participants who can log off -- without consequence -- any time they want. These are precisely the test-takers most sought after by marketers, and they are always present on the Web.

On the other hand, such test participants raise a vital concern among marketers: Are consumers who participate in online tests truly representative of the general public? Or do they reflect only a particular consumer segment, thus leading to highly skewed results?

Demographic studies tell us avid Internet users differ in some ways from offline consumers. Both the elderly and uneducated people are underrepresented among online consumers. Also, some Internet users are early adopters of innovative technology who were lured to the Internet years ago -- which means that for low-tech concepts, online test results may be skewed.

The Internet, then, is the perfect place to conduct a concept test for high-tech products, since the Web is a comfortable milieu for techies. This of course makes the Web a more natural test environment for computers and electronic products than for scrub brushes and spray cleaners.

The cons of online testing are many and varied.

* Nearly everyone is familiar with at least one: hackers. Hackers can create havoc with online tests, but only if they gain access to ideas for new concepts or product blueprints, or are able to tamper with test results -- not an easy task.

But these security risks are by no means peculiar to the Internet, and they do not pose a major barrier for online testing.

* A more pressing concern is how results for tests conducted offline may differ from those conducted online.

Research shows that consumers grade concepts differently online than they do offline. Sometimes, results for purchase interest, uniqueness or believability are lower online. But our experience in concept validation shows that marketing decisions usually are unaffected by these differences. More studies need to be done in this area.

* Another worry involves, ironically, the very factor that's a major benefit in online testing: time. Time savings can be offset in the test design stage, when developing test materials. Questionnaires posted on the Web, for instance, must be worded in ways that elicit desired results, giving testers headaches they may not have experienced in offline tests. Testing for too short a time frame precludes connecting with "light" browsers who could be heavy category users.

In addition, savings in test budgets can be eaten up in the follow-up research to ensure that the test has yielded reasonable numbers.

* The possibility of sabotage can also be a factor. Respondents are often cloaked in such secrecy that marketers may never know who they are, or whether competition is somehow manipulating the test. But this problem is not unique to the Internet; offline tests are subject to similar risks.

Online testing is imperfect in many ways, but marketers are coming to feel the pros outweigh the cons.


The Internet offers meaningful -- perhaps even revolutionary -- ways to create dialogue with consumers, or even regain customer relationships that were once taken for granted but which are seldom experienced today, such as when consumers welcomed marketers into their homes to peek in the kitchen cupboard.

Though online test results can differ slightly from other test methods, in the long run, concept validations show that marketing decisions remain unchanged despite the differences, at least in our experience.

And as for market research applications beyond concept testing, online data will be used in the same ways as offline-generated data, but with a judicious eye kept on potential differences related to the medium and the sample. Eventually, we expect new and more useful measures and perspectives to emerge.

With time, market researchers will learn to distinguish most of the important differences between online and offline testing. But many marketers agree that as the number of Internet users increases, the differences between online and offline test results diminish.

Online market researchers may also agree on another point -- that they sometimes feel as if the clock has been turned back. They often hear doubts and fears about technology that have a similar ring to those faced when the telephone was new, and wire carried voice instead of gigabytes of data.

Mr. Newman is market research manager of the Unilever Interactive Brand Center, based in Englewood Cliffs, N.J.

In this article: