Lessen bias with multiple sources

Published on .

Question: If I use a Web-based approach to do market research what, if any, data quality issues should I be concerned about?

Answer: Data quality is often a function of the approach used. For example, if your research requires that a respondent ponder several long sentences to decide which best characterizes his or her organization, a phone study may be the wrong approach. It's difficult for respondents to remember details from four or five long statements and make an accurate choice. Conversely, if the research demands unaided answers to questions, a phone interview is likely to yield the best quality data. It is extremely important to match the research requirements to the methodology.

Once this decision is made, data quality issues specific to the method selected must be addressed. For our purposes, we'll assume that a Web-based approach was the method of choice. Two of the more critical issues confronting a Web-based study are finding high-quality sample sources and making sure you have minimized the bias inherent in any one source.

Let's take these in reverse order. All sources of potential respondents-unless they represent a true census of all possible respondents-have some bias. The bias is a function of how the potential respondents were recruited into a list or panel. Often the nature of the bias is not known. Therefore, the only way to reduce the likelihood of a bias is to use multiple sources. The assumption is that each source, while overlapping others, contributes uniquely to the universe of possible respondents; the effect is to cancel out the bias in another source.

Finding a good quality sample source is always challenging, and there is no substitute for a formal evaluation of or experience with numerous sources. Each source will be better suited to some research studies than others. One thing is for certain: The old adage "you get what you pay for" will apply more often than not.

Carey Azzara is director of research at Management Insight Technologies, a market research consulting company ( ).

Most Popular
In this article: