While the two slicers and dicers of online audience data-ComScore Networks' ComScore Media Metrix and Nielsen/NetRatings-are criticized for a laundry list of problems, the methods by which they recruit consumers to their at-work panels, as well as the quality and the size of their samples, remain overriding issues.
"The bottom line is the numbers are not right. They're wrong, and decisions are being made on billions of dollars in ad spending against these numbers, which are erroneous," said David Cohen, senior VP-interactive media director, Universal McCann Interactive. Mr. Cohen said comparisons with actual ad-serving data from DoubleClick and Aquantive's Atlas DMT show that both services are off anywhere from 80% to 130% with regard to reach and frequency, calling into question the validity of their data.
Howard Schimmel, VP-market research, at Time Warner's America Online, wants more precise intelligence from the data: "We don't have enough insight or information to make smart programming decisions."
He'd like to see detailed day-part data, next-day ratings similar to those in the TV business and monthly run-of-schedule data. Mr. Schimmel believes both services should be audited and accredited by the Media Ratings Council.
Bruce Rogers, VP-marketing at Forbes.com, subscribes only to ComScore's syndicated research. His problem is that for any given month, ComScore's figures for its traffic are "50% to 100% lower than what Forbes.com site traffic actually is" according to the site's internal log files.
ComScore and Nielsen have different panel recruitment methods. Nielsen recruits by phone via random digit dialing, while ComScore combines that method with online banner ads designed to entice potential panelists with virus-protection software and speedier broadband connections. "If people join for those reasons, you get skewed samples," argued a research director for a major online publisher.
Of the 43,600 individuals in Nielsen's panel as of September, 5,950 were classified as "at-work." By contrast, ComScore's 100,000-member panel comprises at-home Web surfers (50,000), at-work users, which includes home-office workers (30,000), and university/full-time students (20,000). Workplaces often prohibit panel participation and installation of third-party software.
Manish Bhatia, senior VP, Nielsen NetRatings, argued that recruiting for the workplace panel is more complicated than it is for the home panel. "A large panel does not equal more accurate numbers," he added. Mr. Bathia suggested Nielsen's biggest problem remains reconciling comparisons publishers make between their internal Web site log-file data and Nielsen data.
According to ComScore Networks CEO Magid Abraham, ComScore's at-work sample is "just as random as the Nielsen one and more accurately weighted because we're taking into account the size of the company and the proportions of the people we recruit at random." He maintained that Nielsen's at-work sample is too small.
ComScore said it's talking with the MRC, the Internet Advertising Bureau and the Online Publishers Association. "We certainly believe there's a role for the industry bodies to play here, but it's a much more complex process in the Internet space than in the TV world," said Peter Daboll, division president, ComScore Media Metrix.