Consumer Test Markets Fail to Reflect Ethnic Cross-Section

By Published on .

CINCINNATI (AdAge.com) -- In Cedar Rapids, Iowa, there are tree- lined small-town neighborhoods with modest, early 20th century frame houses, a couple of big manufacturing plants and a Class A baseball park (where fans compete to throw plungers through a toilet seat in between-innings promotions for a local plumber). You'll also find a way-above-average number of new and improved consumer products in Cedar Rapids,
Read Ad Age's full coverage of census-related marketing issues.
one of the busiest test markets in the U.S.

What you won't find are many African-Americans, Asians or Hispanics.

Racially skewed test markets?
Cedar Rapids looks like the America painted by Norman Rockwell, rather than the one painted by the 2000 Census. The rapid growth in minority populations reflected in the census barely made a dent in Cedar Rapids' numbers, or in many of the other most popular test market communities.

Minority under-representation is a problem critics say underlies not only test marketing but also other forms of syndicated market research. Careful as they are, managers of test markets may be overlooking something significant: In the census, 22.9% of people in the U.S. identified themselves to be in a race or races other than white. That's a big, diverse minority population.

Information Resources Inc., the biggest providers of in-market testing services in the U.S., operates one of its five BehaviorScan test markets in Cedar Rapids. Of those five markets, only two have substantial Hispanic communities near or above the national average, and none approaches average national levels of African-American or Asian consumers. The B-Scan markets aren't alone. Other popular test locales, such as Columbus, Ohio, Little Rock, Ark., and Evansville, Ind., are also much whiter than the country as a whole.

'Normative' situation
"I think people are going to have to really come to grips with [census] data to come to understand what constitutes a normative test market location," says Jeff Dufresne, until recently general manager-North American juice drinks for Procter & Gamble Co. and now managing director of BrandStorm, a new product development consulting unit of Northlich, Cincinnati. "Just the minority population growth in general is probably happening on a curve that's three to four years ahead of pre-census projections."

Advocates of test marketing acknowledge potential problems with some popular test markets, but say shifting demographics are manageable and sometimes negligible.

"Yes, you consider demographics [in choosing a test market], but it's only one of the criteria," says Valerie Skala, VP-analytic product management and development for IRI. She argues demographics such as race may not even be the most important criterion, making up no more than half of the influences on purchase decisions, according to IRI research. Category and brand purchase patterns that closely resemble national averages are among the most important factors, she says, as are representative retail development.

More similar than different
"People's basic needs are more similar than they are different," Ms. Skala says. "We all want to be clean and to enjoy good-tasting food and to take care of our families. ... When introducing a new product, your first priority should be getting the basic product and positioning right, in a way that has broad appeal and will generate the most sales."

Costs and speed also figure heavily in the decision of where to test.

Such services as IRI's BehaviorScan and ACNielsen Corp.'s Market Decisions, a unit of VNU, let marketers place their products in test almost instantly-without having to sell to local retailers-thanks to agreements with retail chains in selected markets.

IRI also has expensive and hard-to-duplicate infrastructure in its B-Scan markets, including its consumer panels whose members not only scan their purchases but are also available for in-depth attitudinal research regarding new products or advertising approaches. Cable systems in those markets have been wired to deliver different ads to different homes in IRI's panel, allowing observation of the impact of different marketing approaches even within markets as small as Cedar Rapids. Finally, and perhaps just as importantly, such small-town markets are far enough from major media markets to allow for limited, low-cost media buys.

Two test markets recommended
IRI generally recommends marketers use at least two test markets, not only to compare the impact of different demographics but also to cancel out the potential effects of natural disasters, plant closings or other purely local factors.

If demographics may not be the whole story, they still can be important in choosing a test market for many brands and categories, says BrandStorm's Mr. Dufresne. Sunny Delight, a brand Mr. Dufresne oversaw at P&G, is one example of a brand where testing in a disproportionately white market wouldn't make sense in most cases. Sunny Delight consumption among Hispanics is about 2.5 times what would be predicted based simply on Hispanic population levels.

But even brands with below average Hispanic household penetration may be ill served by testing in markets where Hispanics are under-represented, he says. For instance, Hispanic use of financial services, such as checking accounts, is substantially underdeveloped because of the distrust many immigrants had of financial institutions in their home countries. "Second- and third-generation Hispanics represent a huge opportunity for people in financial services," he says, adding that they need to concentrate product development efforts on that segment.

Hispanics underrepresented
Doing that, however, is difficult when test markets and other forms of syndicated market research under-represent Hispanics, says Isabel Valdes, president of Santiago-Valdes Solutions, a Palo Alto, Calif., consulting firm that specializes in the Hispanic market. She recalls one financial services client a few years ago that was making a decision on a service offering in Dallas based on information from a syndicated consumer panel that had data from only four Hispanics in the metro area.

Ms. Valdes says in the late 1990s she was retained by a large consumer products company she declined to identify to evaluate how well IRI, ACNielsen and other providers of syndicated market research reflect the purchase behavior and attitudes of Hispanics and other minorities. She found almost all syndicated data sources-including turnkey test markets, household panels and retail scanner data-significantly under-represent minority consumers, particularly Hispanics who speak only Spanish.

Her research ultimately led her to become a senior project manager for ACNiel-sen's HomeScan panel. She helped ACNielsen develop a 1,300-member His-panic panel in Los Angeles. While that panel gives marketers meaningful insights into the roughly 63% of the U.S. Hispanic population that is of Mexican descent, it still comes up short in tracking consumers of Caribbean descent, she says. So ACNielsen plans to develop new Hispanic panels in New York and Miami over the next two years.

Even the best intentions, however, sometimes run aground on methodology. Ms. Valdes applauds VNU's Nielsen/NetRatings service for developing an online Hispanic panel recently. The problem, she says, is that the panel was recruited entirely in English, effectively excluding Hispanics who speak only Spanish. Her own research shows that the significant difference in levels of acculturation between Hispanics who only speak Spanish and those who are bilingual or English-dominant translates into significant differences in consumption patterns, too.

Most syndicated data sources similarly under-represent Spanish-only Hispanics, Ms. Valdes says, either because they recruit panels in English or, in the case of IRI and ACNielsen syndicated retail data, don't track stores where many Spanish-speaking consumers shop. Neither IRI nor ACNielsen collects scanner data from the bodegas, Asian-owned neighborhood stores or club stores where many minorities spend much of their money.

Because of the shortcomings of syndicated retail data and household panels, marketers might not get a complete read of Hispanic consumers even if they were testing in markets with proportionate levels of Hispanic consumers, Ms. Valdes says.

Even if marketers find a small market that adequately represents the U.S. minority population in numbers, they still might lack a truly representative market, Ms. Valdes says. Since most minority consumers are clustered in large urban markets, small-town markets may not adequately reflect the retail patterns, pricing or preferences of the broader minority market, she says.

"You need to measure the impact where your campaign and marketing efforts have been concentrated," she said, "not just where you happened to find people to make a quota." Even if consumer panels in small test markets were balanced with more minority consumers, many of those markets still don't have Spanish-language TV, such as the Univision or Tele-mundo networks.

Gisele Simmons, principal of the multicultural marketing consulting firm Gisele Simmons & Associates, Spring Valley, N.Y., agrees that most test markets don't fully reflect the impact of African-American consumers. Even in markets that are statistically representative, scanner data doesn't capture purchases made in many small, independent stores that don't use scanners but where many African-Americans do much of their shopping, she says.

"If you are looking at traditional methodologies to determine how you are going to apply your dollars in advertising, marketing and targeting, [lack of representation of African-Americans] definitely has an impact," she says, noting that many test market locales lack significant penetration of ethnic media. She estimates about 50% of the African-American population lives in the Southeast, which has relatively few test markets.

While experts may fret over demographic fine tuning to find optimal ways of testing products in the U.S., some marketers are basing rollout decisions on data that includes no U.S. consumers or media. P&G, H.J. Heinz Co. and M&M/Mars are among marketers that have used results from rollouts in Canada or Europe to make decisions about U.S. launches.

P&G Treasurer Gretchen Price told financial analysts last year that P&G, under its new global marketing management structure, has found that global markets tend to fall into clusters with similar consumption and attitude patterns that can be used for testing.

The U.S., for instance, is grouped with Canada, the United Kingdom and the Benelux countries in a sort of common market, which has allowed P&G to base launch plans in the U.S. on results from such markets as Belgium and Canada.

P&G is backing Swiffer WetJet in the U.S. in August, with TV and print ads from Bcom3 Group's D'Arcy Masius Benton & Bowles, following "off-Broadway" runs in Belgium and Canada that began last year.

Overseas tests, however, can be hard to project in the U.S., particularly in food, but also in some household products, says Tom Vierhile, general manager of Marketing Intelligence Service, a firm that tracks new products in the U.S. and globally.

He points to laundry detergent tablets, which captured 10% to 20% of some European markets, including the U.K., in their first year, but have been far less successful in the U.S. since their launch last fall. Unilever acknowledged last month that the launch of its Wisk tablets had been disappointing, with overall tablet sales capturing only about 2% of the $4.6 billion U.S. laundry detergent market so far (though Unilever said it's ahead of P&G's Tide tablets in trial and repeat rates).

"You have to be careful about making the assumption that everything is the same everywhere, because it's not," Mr. Vierhile said. "And some of those things can trip companies up."

Most Popular
In this article: