No doubt about it: Online testing can be highly risky. But even conservative package-goods companies are embracing such methods in record numbers as the breakneck race to market becomes ever more intense.
Used as a faster, less expensive alternative to traditional tests, the Internet has opened up a new forum for everything from focus groups to real-world market simulation.
"There's a lot of pressure to move as fast as you can with the best possible information," said Ellen Gottlich, associate director of consumer understanding for Unilever's Home & Personal Care unit. That's a leap for an industry where P&G, for example, tested Febreze, Dryel and Fit Fruit & Vegetable Wash for more than five years before launching them nationally. Compare that to the same company's Crest MultiCare Flex & Clean, which rolled out this fall, less than a year after online tests.
But while the advantage of speed is clear-Ms. Gottlich said online focus groups can shrink testing time from the usual two to three weeks to one-it comes with added uncertainty.
"Certainly we are concerned about a brand new product concept ending up in the hands of a competitor" said Bill Reynolds, director of marketing services at Unilever Home & Personal Care. "We have weighed in with that concern [to our vendors], and I'm sure we're not alone in that area."
In the case of the Crest fiasco, Charles Hamlin, president of InsightExpress-an NFO Worldwide unit that shares an Internet server with the NFO//net.discussion service from which the P&G concepts were leaked-said someone hacked into the server to find and post links to the images. He added that NFO, which conducted the P&G tests, is investigating.
`AN ISOLATED INCIDENT'
"This is really a one-time, isolated incident, and steps have been made to ensure it doesn't happen again," said Bryan McCleary, supervisor-oral care, public relations, at P&G. "We strongly believe that Internet testing is a wonderful advance and a powerful new way of getting to know our consumers and that the benefits strongly outweigh the risks."
Research industry observers have long seen risks in sending sensitive concept images to the computer screens of consumers, but the usual fear isn't hackers. "You just honestly can't [provide complete security] on the Web," said Dan Coates, VP-consumer intelligence of online marketing feedback site Planetfeedback.com, and a founder of two interactive market research units. "For every technology we would try to deploy to protect the concepts, there was another technology that could surpass it."
But other testing forms, such as mail, pose risks as great, Mr. Hamlin said. And he doesn't believe security concerns will thwart rapid growth of online testing, which he expects to account for 25% of consumer market research by 2002, up from less than 1% currently.
Lower costs also could be a factor in that rise. Offline testing can cost anywhere from $2,000 for a focus group to more than $25,000 for a concept test run in a shopping mall, Mr. Coates said. When marketers test dozens or hundreds of concepts a year, it adds up.
Marketers, however, are wrestling with the issue of determining how well Internet users represent consumers generally, and whether online tests deliver results as reliable as offline forms.
"We know from years of validation that mall-intercept testing or consumer panel testing correlates with real-world purchase data," said Doug Hall, founder and president of new-products consultancy Richard Saunders International. "But we know that online [test results] don't correlate with mall intercept and there's no evidence that they correlate with consumer purchases."
Aware of such concerns, Unilever is "in the process of completing a number of validations [of online consumer testing] and the results look fairly good," Ms. Gottlich said.
She acknowledged that Web users score new concepts lower for uniqueness than do consumers tested offline, and that distribution of data is different. But in the end, she said, Unilever executives appear to be making the same decisions using online tests that they would if they used offline testing.
Even with the growth of the online population, it's too soon to conclude that Web users accurately reflect consumers generally, research executives believe. Online researchers are trying various ways around the problem, including InterSurvey, a Palo Alto, Calif., company that's putting free WebTV Internet access into a representative sample of households that agree to take surveys regularly.
But people who agree to take Web surveys tend to be more Web-savvy than others online, one research executive said, meaning that online researchers are getting a relatively sophisticated segment of the online population.
Even the same consumers tend to score concepts differently online than off, according to research from Burke Marketing. That research found consumers grade concepts closer to the middle range online and more at the high or low ends in verbally directed surveys.
Those middle-range responses are probably closer to how consumers really feel, said Jeff Miller, exec VP of Burke. But they can present new challenges in calibrating tests for accurate decision-making and in comparing online results with offline databases, Mr. Coates said.
On the positive side, Burke also found Web participants enjoy taking more surveys, finish faster and are more likely to repeat the process than are offline participants, Mr. Miller said.
As a solution to concerns about Internet testing-as well as to increase speed and lower costs-former P&G executive Mr. Hall has developed Merwyn, software that forecasts consumer acceptance of new products and services using a database of 4,000 concepts that has been tested over the past two years by such companies as ACNielsen Corp.'s Bases, NPD Group and AcuPOLL International.
Merwyn analyzes consumer purchase intent for concepts based on how well they measure up to what past consumer tests showed to be "the laws of marketing physics," Mr. Hall said.
Those laws state that the most successful concepts are those that convey overt benefits, real reasons to believe in those benefits and dramatic differences from existing products or services. The software also suggests ways marketers can improve a concept's scores.
Like online market researchers, Mr. Hall bills his product more as a way to screen preliminary concepts than test final ones. But he claims that running hundreds of concepts through Merwyn to find the handful that merit further development can whittle to as little as a day a process that Andersen Consulting estimates now takes 17 weeks.
Using simulation of a different sort, Information Resources Inc. last year launched IntroCast, which can trim time off test markets by forecasting a new product's volume based on past products' results.
IRI clients are using IntroCast to monitor whether a new product's trial and repeat purchase numbers are on pace to meet long-term goals and gauge how various levels of advertising and promotion will affect volume. Using IntrroCast can trim the $1 million cost of a traditional in-market test by about 25% and, in some cases, up to 50%.
"With most new products, we can [use IntroCast to] do a decent job of forecasting year-one [national sales] potential after 12 to 16 weeks [in a test market]," Mr. Findley said, a quarter of the time required for a traditional test.
Such speed translates into lower cost and less risk of competitors discovering test markets in time to monitor or disrupt them, he said. IntroCast "also lets you play with alternative marketing plans," Mr. Findley said, without having to use simultaneous tests in multiple cities.
IRI client P&G, which in the past often ran tests in two or more cities to try different marketing support levels, is using just one BehaviorScan market each for such products as Impress plastic wrap and Bounty napkins, though Mr. Findley didn't comment on whether these moves are linked to IntroCast.
Not every product is ripe for simulation, however, Mr. Findley acknowledged. Repeat purchase rates are difficult to project for products in new categories, such as P&G's Dryel and Swiffer, so marketers generally either have to test longer or supplement their testing with consumer surveys. (While P&G's rival S.C. Johnson & Son didn't test its Swiffer follow-up, Pledge Grab-It, it can be argued that the company had the benefit of monitoring P&G's test results for Swiffer).
IntroCast will likely mean shorter test markets in fewer cities, but the software is still no replacement for in-market testing, Mr. Findley said, adding: "Test marketing is alive and well."
Unilever's Mr. Reynolds however, believes marketers will keep looking for ways around test marketing.
"If it's a high-risk, high-return idea in one of your key strategic categories, that would argue for deeper testing," Mr. Reynolds said. "But because of the cost and time and visibility to the competition, there will be a general trend to less in-market testing."