Packaged-Goods Testing Gets a Makeover
Traditional product testing told Big Food marketers Greek yogurt was a small idea and threatened to strangle Procter & Gamble Co.'s Swiffer in its cradle by underestimating repeat purchases, say packaged-goods veterans.
Ultimately, Chobani made a millionaire of entrepreneurial founder Hamdi Ulukaya, and Swiffer became a brand with retail sales of more than $700 million annually in the U.S. alone -- solid proof the tests were wrong. Even when they were right, they were often slower and more expensive than marketers wanted. So now the tests themselves are getting a makeover.
SymphonyIRI recently closed its BehaviorScan test markets products in middle-American enclaves Cedar Rapids, Iowa; Eau Claire, Wis.; Grand Junction, Colo.; and Pittsfield, Mass. For more than three decades, those towns, wired to serve different ads to different homes and track purchases by special consumer panels, served as the final testing grounds for thousands of products on their way to store shelves -- or to the dustbins of CPG history.
Another traditional standard of CPG product and concept testing, Nielsen Bases, also hit turbulence recently. The Insights business within Nielsen's "Buy" division, where Bases is housed, saw sales decline 9% in the third quarter, something Nielsen CEO David Calhoun in an earnings conference call blamed on clients delaying projects due to economic concerns, though he said the biggest clients are spending more on such tests.
There are some bright spots in the simulated test-market industry, however. At least one Bases competitor, Cincinnati-based AcuPoll, saw "a record year," said CEO Jeffrey Goldstein, though he said all marketers are looking to do their testing faster, cheaper and more accurately.
And a small challenger to now-defunct BehaviorScan markets is seeing rapid growth by promising the speed and relative confidentiality of survey-based tests like Bases with the real-world accuracy of in-store tests.
GameChanger, a San Francisco company that specializes in tests that can encompass anything from a weekend in one store to several weeks in around a dozen stores (or even tests done online via e-commerce), saw sales grow more than 40% last year to $4.5 million. That came after a 518% sales hike the prior year, which landed it in the Inc. 500.
That's not entirely coincidental. GameChanger Director Laura King was global brand manager on Swiffer's launch in the late 1990s. She recalls Bases testing at the time predicted a low repeat rate that would have Swiffer generate $20 million to $40 million in first-year sales. That would have kept Swiffer from meeting P&G's $100 million hurdle rate, she said.
The brand team's own experience with enthusiastic test users suggested a higher repeat rate. And Mr. Jager, who was chief operating officer at P&G at the time, greenlighted the launch anyway, Ms. King said.
Mr. Jager said in an email that he didn't recall those details on Swiffer, but said he already had concluded, based on a Downy product that fared far better than its pre-market testing indicated, that it's "difficult to predict in-market results of a new-to-the-world benefit without going through real market tests."
Mike Paul, a 16-year veteran of General Mills who spent five years working in Yoplait product development before leaving to start the consultancy Vision Quest Innovation in 2011, became a convert to GameChanger's small-scale in-store tests while at General Mills. Part of that conversion came from Greek yogurt.
Bases, or General Mills' proprietary version of it, Mr. Paul said, doesn't do well at testing things that differ too much from what consumers are used to. "Greek yogurt tested horribly," he said. "Consumers didn't have a precedent."
General Mills did not immediately respond to a request for comment.
Even with tests at only a dozen stores, GameChanger can use targeted TV, digital and other media, like BehaviorScan tests once did, Mr. Popelka said. But the tests happen in varied locations to make them harder for competitors to find, with brands and marketer names sometimes changed to avoid detection via scanner data or media trackers such as Kantar Media and Nielsen.
The in-store tests range from as low as $50,000 for one store on a weekend to $350,000 for tests over multiple weeks at a dozen or more stores with a full marketing plan, which Mr. Popelka said was similar to Bases (Nielsen declined to comment on costs). But Mr. Popelka said his setup time is six to eight weeks vs. six to 12 months and $1 million to $2 million for BehaviorScan or larger-scale test markets companies do themselves.
Bases is far from obsolete and is even gaining market share, said Rob Wengel, senior VP of the Nielsen unit. But many big marketers are changing their innovation processes to speed them up and seeking "fewer, bigger" innovations.
"We've not seen a decline in the volume of tests," Mr. Wengel said. "We're seeing a lot more work done in the earlier stages."
He said he believes in-store testing is declining compared to simulated test markets like Bases, which, he said, "are not only much more cost effective and confidential, but also much more predictive of enduring success."
For its part, SymphonyIRI hasn't given up on in-store testing. It still does tests in smaller groups of stores around the country, said Robert Tomei, president-consumer and shopper marketing. And the company does online "virtual shopping" tests with consumers that simulate the in-store experience.
"The testing services have evolved pretty dramatically with the advent of virtual and digital, and results coming in days and weeks vs. weeks and months," said SymphonyIRI Chief Marketing Officer John McIndoe. "So we've evolved our testing services."
