That's not entirely coincidental. GameChanger Director Laura King
was global brand manager on Swiffer's launch in the late 1990s. She
recalls Bases testing at the time predicted a low repeat rate that
would have Swiffer generate $20 million to $40 million in
first-year sales. That would have kept Swiffer from meeting
P&G's $100 million hurdle rate, she said.
The brand team's own experience with enthusiastic test users
suggested a higher repeat rate. And Mr. Jager, who was chief
operating officer at P&G at the time, greenlighted the launch
anyway, Ms. King said.
Mr. Jager said in an email that he didn't recall those details
on Swiffer, but said he already had concluded, based on a Downy
product that fared far better than its pre-market testing
indicated, that it's "difficult to predict in-market results of a
new-to-the-world benefit without going through real market
Mike Paul, a 16-year veteran of General Mills who
spent five years working in Yoplait product development before
leaving to start the consultancy Vision Quest Innovation in 2011,
became a convert to GameChanger's small-scale in-store tests while
at General Mills. Part of that conversion came from Greek
Bases, or General Mills' proprietary version of it, Mr. Paul
said, doesn't do well at testing things that differ too much from
what consumers are used to. "Greek yogurt tested horribly," he
said. "Consumers didn't have a precedent."
General Mills did not immediately respond to a request for
Even with tests at only a dozen stores, GameChanger can use
targeted TV, digital and other media, like BehaviorScan tests once
did, Mr. Popelka said. But the tests happen in varied locations to
make them harder for competitors to find, with brands and marketer
names sometimes changed to avoid detection via scanner data or
media trackers such as Kantar Media and Nielsen.
The in-store tests range from as low as $50,000 for one store on
a weekend to $350,000 for tests over multiple weeks at a dozen or
more stores with a full marketing plan, which Mr. Popelka said was
similar to Bases (Nielsen declined to comment on costs). But Mr.
Popelka said his setup time is six to eight weeks vs. six to 12
months and $1 million to $2 million for BehaviorScan or
larger-scale test markets companies do themselves.
Bases is far from obsolete and is even gaining market share,
said Rob Wengel, senior VP of the Nielsen unit. But many big
marketers are changing their innovation processes to speed them up
and seeking "fewer, bigger" innovations.
"We've not seen a decline in the volume of tests," Mr. Wengel
said. "We're seeing a lot more work done in the earlier
He said he believes in-store testing is declining compared to
simulated test markets like Bases, which, he said, "are not only
much more cost effective and confidential, but also much more
predictive of enduring success."
For its part, SymphonyIRI hasn't given up on in-store testing.
It still does tests in smaller groups of stores around the country,
said Robert Tomei, president-consumer and shopper marketing. And
the company does online "virtual shopping" tests with consumers
that simulate the in-store experience.
"The testing services have evolved pretty dramatically with the
advent of virtual and digital, and results coming in days and weeks
vs. weeks and months," said SymphonyIRI Chief Marketing Officer
John McIndoe. "So we've evolved our testing services."