Online Research: Don't Confuse More With Better

Common Pitfalls and Ways to Improve the Process

By Published on .

Constance O'Hare
Constance O'Hare
The internet has changed the way we do research. Sure it's cheap and fast, but is the ability to get instant data actually making executives smarter about market opportunities? Not always. In too many companies, online research creates an illusion of rigor while actually sowing confusion about market truths, leading executives -- in particular CEOs and CMOs -- to miss the big picture and waste opportunities right under their noses.

The ease of fielding multiple online studies naturally appeals to executives grasping for insights and under pressure to make decisions quickly. But too many impatient executives throw away money on poorly constructed studies. Studies that produce speedy results but ask the wrong questions, ask the wrong people, or ask in the wrong way waste resources and time -- and lead to bad market strategy. Here are some common pitfalls:

Less thought because it's "cheap and fast." Many companies overlook what it takes to get good answers and fall prey to the dangers of panel bias. Online panels typically attract white, rich, well-educated participants who provide few insights into the needs and preferences of growing populations, such as value-oriented, lower income, Hispanic consumers. In recent work studying the market for wireless services, we had to recruit almost three times the number of respondents relative to the sample desired to represent this broadly diverse pool of consumers.

Inadequate quality control. Is it easier to lie online? Sending researchers out into the field has become increasingly rare. Too many companies now skip the vital qualitative research that's essential to the design of valid quantitative research instruments and helps ensure that survey data will be meaningful. Omitting this important first step makes it impossible to ensure that the data collected are "clean," that all respondents are unique and qualified, and that they're taking the time to answer with care rather than simply "straight-lining."

Piecemeal studies by department. Without strong CEO or CMO leadership, individual departments will use research to pull for their own focus or justify existing initiatives, especially during tough times. Such fragmentation makes "common knowledge" harder to achieve and can undercut a company's overall performance. Strong brand growth relies on a shared view of market realities, challenges and opportunities. At a minimum, common data are essential to prevent internal politics from driving bad decision-making.

'Dip-stick' vs. longitudinal tracking. How can a company know what to do to excel in six months, a year? Landmark studies, such as Nielsen's tracking of American's TV-viewing habits, track market trends in motion. By contrast, dip-stick studies, or snapshots of one point in time, do little to help companies predict change. For example, traditional grocery chains have missed opportunities in convenience food by looking only at existing grocery spending even though we now know that over a third of consumers would buy takeout from a grocery store and takeout can represent 20% of all food sales.

Loss of strategic perspective. Especially in the largest companies, one-off studies become distractions for leaders who are trying to get "strategically centered." You'll hear research managers (as we did in one Fortune 50 healthcare company) boast about "dozens of projects" underway while top executives struggle with fundamental questions about how to position the brand.

So how do you improve the process? The answer is not to get better at interpreting the findings of the wrong studies. The answer is to do fewer studies -- but do them right.

Spend money on answering the big questions. Stop the piecemeal studies. Have you identified the key unknowns that matter to overall performance? Why and how much do consumers buy, and from whom? Where, specifically, are competitors vulnerable? What short-term moves will yield the highest return? Without such information, a company can miss sizable opportunities while focused competitors erode its position.

Harvest "free" data pouring in about your company. Before wasting another dime on a satisfaction survey, mine what customers are saying to you and about you through existing channels: the call center, website, YouTube, Twitter, Facebook, blogs. The customer-service experience is a key driver of positive or negative brand impressions. So why not take a valid sample of customer calls and examine who is calling and why? Working with a very large cable company, we discovered not only that frequent callers were high-value customers (paying monthly bills 30% above average) but that 50% of the time, their calls were not resolved the first time. No wonder these customers were being lost to the competition.

Invest in an ongoing, full-market market panel. A market panel is a sweeping, detailed and continuing survey of a large and carefully selected group of buyers who reflect a statistically reliable sample of a much larger market. An antidote to fragmentation, a market panel provides a wide-angle view of a market's total buying population and provides a series of individual portraits every six to 12 months that are designed to help a company predict shifts. With the steady decline of brand loyalty in many market segments and the emergence of new and complex shopping behaviors driven by the web, tracking buyers over time is critical -- especially to improving the value of advertising.

Distribute the insights companywide: Raise your company IQ. Prevent concealment and empire-building. Money spent on market research should be seen as a shared investment in raising company intelligence.

Beyond wasting time and dollars, a preoccupation with "more" research impedes strategic thinking, making it harder for discerning CEOs and CMOs to chart the right course with conviction. Don't ask your agency to work miracles. It's your job as CEO or CMO to develop an intelligent and compelling view of your own market. Make sure every dollar you spend advances your company's competitive position. Invest in knowledge rather than in a mountain of studies that can produce varying and conflicting results.

Constance O'Hare is owner-managing partner of Beacon Advisors, a Boston-based strategy firm. She is currently on the editorial advisory board of MIT Sloan Management Review and is an instructor in strategic management at Harvard University.
Most Popular
In this article: